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Electrical engineering education has undergone some radical changes during the past cou- 
ple of decades and continues to do so. A modern undergraduate program in electrical 
engineering includes the following two introductory courses: 


> Signals and Systems, which provides a balanced and integrated treatment of contin- 
uous-time and discrete-time forms of signals and systems. The Fourier transform (in 
its different forms), Laplace transform, and z-transform are treated in detail. Typi- 
cally, the course also includes an elementary treatment of communication systems. 

> Probability and Random Processes, which develops an intuitive grasp of discrete and 
continuous random variables and then introduces the notion of a random process 
and its characteristics. 


Typically, these two introductory courses lead to a senior-level course on communication 
systems. 

The fourth edition of this book has been written with this background and primary 
objective in mind. Simply put, the book provides a modern treatment of communication 
systems at a level suitable for a one- or two-semester senior undergraduate course. The 
emphasis is on the statistical underpinnings of communication theory with applications. 

The material is presented in a logical manner, and it is illustrated with examples, 
with the overall aim being that of helping the student develop an intuitive grasp of the 
theory under discussion. Except for the Background and Preview chapter, each chapter 
ends with numerous problems designed not only to help the students test their understand- 
ing of the material covered in the chapter but also to challenge them to extend this material. 
Every chapter includes notes and references that provide suggestions for further reading. 
Sections or subsections that can be bypassed without loss of continuity are identified with 
a footnote. ; 

A distinctive feature of the book is the inclusion of eight computer experiments using 
MATLAB. This set of experiments provides the basis of a “Software Laboratory”, with 
each experiment being designed to ead the material covered in the pertinent chapter. 
Most important, the experiments exploit the unique capabilities of MATLAB inan instruc- 
tive manner. The MATLAB codes for all these experiments are available on the Wiley Web 
site: http://www.wiley.com/college/haykin/. 

The Background and Preview chapter presents introductory and motivational ma- 
terial, paving the way for detailed treatment of the many facets of communication systems 
in the subsequent 10 chapters. The material in these chapters is organized as follows: 


> Chapter 1 develops a detailed treatment of random, or stochastic, processes, with 
particular emphasis on their partial characterization (i.e., second-order statistics). In 
effect, the discussion is restricted to wide-sense stationary processes. The correlation 
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properties and power spectra of random processes are described in detail. Gaussian 
processes and narrowband noise feature prominently in the study of communication 
systems, hence their treatment in the latter part of the chapter. This treatment nat- 
urally leads to the consideration of the Rayleigh and Rician distributions that arise 
in a communications environment. 
Chapter 2 presents an integrated treatment of continuous-wave (CW) modulation 
(i.e., analog communications) and their different types, as outlined here: 
(i) Amplitude modulation, which itself can assume one of the following forms (de- 
pending on how the spectral characteristics of the modulated wave are specified): 
» Full amplitude modulation 
® Double sideband-suppressed carrier modulation 
> Quadrature amplitude modulation 
» Single sideband modulation 
> Vestigial sideband modulation 
(ii) Angle modulation, which itself can assume one of two interrelated forms: 
» Phase modulation 
> Frequency modulation 
The time-domain and spectral characteristics of these modulated waves, methods for 
their generation and detection, and the effects of channel noise on their performances 
are discussed. 
Chapter 3 covers pulse modulation and discusses the processes of sampling, quan- 
tization, and coding that are fundamental to the digital transmission of analog sig- 
nals. This chapter may be viewed as the transition from analog to digital commu- 
nications. Specifically, the following types of pulse modulation are discussed: 
(i) Analog pulse modulation, where only time is represented in discrete form; it 
embodies the following special forms: 
® Pulse amplitude modulation 
> Pulse width (duration) modulation 
œ Pule position modulation 
The characteristics of pulse amplitude modulation are discussed in detail, as it is 
basic to all forms of pulse modulation, be they of the analog or digital type. 
(ii) Digital pulse modulation, in which both time and signal amplitude are repre- 
sented in discrete form; it embodies the following special forms: 
= Pulse-code modulation 
= Delta modulation 
» Differential pulse-code modulation 
In delta modulation, the sampling rate is increased far in excess of that used in pulse- 
code modulation so as to simplify implementation of the system. In contrast, in 
differential pulse-code modulation, the sampling rate is reduced through the use of 
a predictor that exploits the correlation properties of the information-bearing signal. 


(iii) MPEG/audio coding standard, which includes a psychoacoustic model as a key 
element in the design of the encoder. 


Chapter 4 covers baseband pulse transmission, which deals with the transmission of 
pulse-amplitude modulated signals in their baseband form. Two important issues are 
discussed: the effects of channel noise and limited channel bandwidth on the perfor- 
mance of a digital communication system. Assuming that the channel noise is additive 
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and white, this effect is minimized by using a matched filter, which is basic to the 

design of communication receivers. As for limited channel bandwidth, it manifests 

itself in the form of a phenomenon known as intersymbol interference. To combat 
the degrading effects of this signal-dependent interference, we may use either a pulse- 
shaping filter or correlative encoder/decoder; both of these approaches are discussed. 

The chapter includes a discussion of digital subscriber lines for direct communication 

between a subscriber and an Internet service provider. This is followed by a deriva- 

tion of the optimum linear receiver for combatting the combined effects of channel 
noise and intersymbol interference, which, in turn, leads to an introductory treatment 
of adaptive equalization. 

Chapter 5 discusses signal-space analysis for an additive white Gaussian noise chan- 

nel. In particular, the foundations for the geometric representation of signals with 

finite energy are established. The correlation receiver is derived, and its equivalence 
with the matched filter receiver is demonstrated. The chapter finishes with a discus- 
sion of the probability of error and its approximate calculation. 

Chapter 6 discusses passband data transmission, where a sinusoidal carrier wave is 

employed to facilitate the transmission of the digitally modulated wave over a band- 

pass channel, This chapter builds on the geometric interpretation of signals presented 
in Chapter 5. In particular, the effect of channel noise on the performance of digital 
communication systems is evaluated, using the following modulation techniques: 
(i) Phase-shift keying, which is the digital counterpart to phase modulation with 
the phase of the carrier wave taking on one of a prescribed set of discrete values. 
(ii) Hybrid amplitude/phase modulation schemes including quadrature-amplitude 
modulation (QAM), and carrierless amplitude/phase modulation (CAP). 

(iii) Frequency-shift keying, which is the digital counterpart of frequency modulation 
with the frequency of the carrier wave taking on one of a prescribed set of discrete 
values, 

(iv) Generic multichannel modulation, followed by discrete multitone, the use of 
which has been standardized in asymmetric digital subscriber lines. 

In a digital communication system, timing is everything, which means that the re- 

ceiver must be synchronized to the transmitter. In this context, we speak of the 

receiver being coherent or noncoherent. In a coherent receiver, provisions are made 
for the recovery of both the carrier phase and symbol timing. In a noncoherent 
receiver the carrier.phase is ignored and provision is only made for symbol timing. 

Such a strategy is dictated by the fact that the carrier phase may be random, making 

phase recovery a costly proposition. Synchronization techniques are discussed in the 

latter part of the chapter, with particular emphasis on discrete-time signal processing. 

Chapter 7 introduces spread-spectrum modulation. Unlike traditional forms of mod- 

ulation discussed in earlier chapters, channel bandwidth is purposely sacrificed in 

spread-spectrum modulation for the sake of security or protection against interfering 
signals, The direct-sequence and frequency-hop forms of spread-spectrum modula- 
tion are discussed. 

Chapter 8 deals with multiuser radio communications, where a multitude of users 

have access to a common radio channel. This type of communication channel is well 

represented in satellite and wireless communication systems, both of which are dis- 
cussed, The chapter includes a presentation of link budget analysis, emphasizing the 
related antenna and propagation concepts, and noise calculations. 

Chapter 9 develops the fundamental limits in information theory, which are embod- 

ied in Shannon’s theorems for data compaction, data compression, and data trans- 
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mission. These theorems provide upper bounds on the performance of information 
sources and communication channels. Two concepts, basic to formulation of the 
theorems, are (1) the entropy of a source (whose definition is analogous to that of 
entropy in thermodynamics), and (2) channel capacity. 

> Chapter 10 deals with error-control coding, which encompasses techniques for the 
encoding and decoding of digital data streams for their reliable transmission over 
noisy channels, Four types of error-control coding are discussed: 

(i) Linear block codes, which are completely described by sets of linearly indepen- 
dent code words, each of which consists of message bits and parity-check bits. 
The parity-check bits are included for the purpose of error control. 

(ii) Cyclic codes, which form a subclass of linear block codes. 

(iii) Convolutional codes, which involve operating on the message sequence contin- 
uously in a serial manner. 

(iv) Turbo codes, which provide a novel method of constructing good codes that 
approach Shannon’s channel capacity in a physically realizable manner. 

Methods for the generation of these codes and their decoding are discussed. 


The book also includes supplementary material in the form of six appendices as 
follows: 


> Appendix 1 reviews probability theory. 

> Appendix 2, on the representation of signals and systems, reviews the Fourier trans- 
form and its properties, the various definitions of bandwidth, the Hilbert transform, 
and the low-pass equivalents of narrowband signals and systems. 

> Appendix 3 presents an introductory treatment of the Bessel function and its modified 

form. Bessel functions arise in the study of frequency modulation, noncoherent de- 
tection of signals in noise, and symbol timing synchronization. 

> Appendix 4 introduces the confluent hypergeometric function, the need for which 
arises in the envelope detection of amplitude-modulated signals in noise. 

% Appendix 5 provides an introduction to cryptography, which is basic to secure 
communications. 

> Appendix 6 includes 12 useful tables of various kinds. 


As mentioned previously, the primary purpose of this book is to provide a modern 
treatment of communication systems suitable for use in a one- or two-semester under- © 
graduate course at the senior level. The make-up of the material for the course is naturally 
determined by the background of the students and the interests of the teachers involved. 
The material covered in the book is both broad and deep enough to satisfy a variety of 
backgrounds and interests, thereby providing considerable flexibility in the choice of 
course material. As an aid to the teacher of the course, a detailed solutions manual for all 
the problems in the book is available from the publisher. 


i Acknowledgments 


I wish to express my deep gratitude to Dr. Gregory J. Pottie (University of California, Los 
Angeles), Dr. Santosh Venkatesh (University of Pennsylvania), Dr. Stephen G. Wilson (Uni- 
versity of Virginia), Dr. Gordon Stüber (Georgia Institute of Technology), Dr. Venugopal 
Veeraralli (Cornell University), and Dr. Granville E. Ott (University of Texas at Austin) 


PREFACE xi 


for critical reviews of an earlier version of the manuscript and for making numerous sug- 
gestions that have helped me shape the book into its present form. The treatment of the 
effect of noise on envelope detection presented in Chapter 2 is based on course notes made 
available to me by Dr. Santosh Venkatesh, for which I am grateful. I am grateful to Dr. 
Gordon Stiiber for giving permission to reproduce Figure 6.32. 

I am indebted to Dr. Michael Moher (Communications Research Centre, Ottawa) 
for reading five chapters of an earlier version of the manuscript and for making many 
constructive comments on turbo codes. I am equally indebted to Dr. Brendan Frey (Uni- 
versity of Waterloo, Ontario) for his invaluable help in refining the material on turbo 
codes, comments on low-density parity-check codes, for providing the software to plot 
Fig. 9.18, and giving me the permission to reproduce Figures 10.27 and 10.33. I am grate- 
ful to Dr. David Conn (McMaster University, Ontario) for his critical reading of the Back- 
ground and Preview Chapter and for making suggestions on how to improve the presen- 
tation of the material therein. 

I also wish to thank Dr. Jean-Jacque Werner (Lucent Technologies, Holmdel), Dr. 
James Mazo (Lucent Technologies, Murray Hill), Dr. Andrew Viterbi (Qualcom, San Di- 
ego), Dr, Radford Neal (University of Toronto, Ontario), Dr. Yitzhak (Irwin) Kalet (Tech- 
nion, Israel), Dr. Walter Chen (Motorola), Dr. John Cioffi (Stanford University), Dr. Jon 
Mark (University of Waterloo, Ontario), and Dr. Robert Dony (University of Guelph, 
Ontario); I thank them all for their helpful comments on selected sections in the book. 
Corrections and suggestions for improvements to the book made by Dr. Donald Wunsch 
II (University of Missouri) are also appreciated. 

I am grateful to my graduate student Mathini Sellathurai (McMaster University) for 
performing the computer experiments in the book, and Hugh Pasika (McMaster Univer- 
sity) for many useful comments on the Background and Preview Chapter and for doing 
the computations on some graphical plots in the book. Proofreading of the page proofs 
by Mathini Sellathurai and Alpesh Patel is much appreciated. 

I am particularly grateful to my editor at Wiley, Bill Zobrist, for his strong support 
and help throughout the writing of the book. I am indebted to Monique Calello, Senior 
Production Editor at Wiley, for her tireless effort in overseeing the production of the book 
in its various stages. I thank Katherine Hepburn for advertising and marketing the book. 
I thank Karen Tongish for her careful copyediting of the manuscript, Katrina Avery for 
her careful proofreading of the page proofs, and-Kristen Maus for composing the index 
of the book. 

Last but by no means least, as always, I am grateful to my Technical Coordinator, 
Lola Brooks, for her tireless effort in typing the manuscript of the book. I also wish to 
record my gratitude to Brigitte Maier, Assistant Librarian, and Regina Bendig, Reference 
Librarian, at McMaster University, for helping.me on numerous occasions in tracing ref- 
erences for the bibliography. 


Simon Haykin 
Ancaster, Ontario 
January, 2000 





i BACKGROUND AND PREVIEW i 1 


G: 


a a 


10. 


The Communication Process 1 

Primary Communication Resources 3 
Sources of Information 3 

Communication Networks 10 
Communication Channels 15 

Modulation Process 19 

Analog and Digital Types of Communication 21 
Shannon’s Information Capacity Theorem 23 
A Digital Communication Problem 24 
Historical Notes 26 

Notes and References 29 


i CHAPTER | Random Processes 31 


1.1 
1.2 
1.3 
1.4 
1.5 
1.6 
1.7 
1.8 
1.9 
1.10 
1.11 


1.12 


1.13 
1.14 


Introduction 31 

Mathematical Definition of a Random Process 32 

Stationary Processes 33 

Mean, Correlation, and Covariance Functions 35 

Ergodic Processes 41 

Transmission of a Random Process Through a Linear Time-Invariant Filter 42 
Power Spectral Density 44 

Gaussian Process 54 

Noise 58 

Narrowband Noise 64 


Representation of Narrowband Noise in Terms of In-phase and Quadrature 
Components 64 À 


Representation of Narrowband Noise in Terms of Envelope and Phase 
Components 67 


Sine Wave Plus Narrowband Noise 69 
Computer Experiments: Flat-Fading Channel 71 


xiv CONTENTS 


1,15 


CHAPTER 2 


Summary and Discussion 75 
Notes and References 77 
Problems 78 


Continuous-Wave Modulation 88 





2.1 
2.2 
2.3 
2.4 
2.5 
2.6 
2.7 
2.8 
2.9 
2.10 
2.11 
2.12 
2.13 
2.14 
2.15 


Introduction 88 
Amplitude Modulation 90 


_ Linear Modulation Schemes 93 


Frequency Translation 103 

Frequency-Division Multiplexing 105 

Angle Modulation 107 

Frequency Modulation 109 

Nonlinear Effects in FM Systems 126 
Superheterodyne Receiver 128 

Noise in CW Modulation Systems 130 

Noise in Linear Receivers using Coherent Detection 132 
Noise in AM Receivers using Envelope Detection 135 
Noise in FM Receivers 142 

Computer Experiments: Phase-locked Loop 157 
Summary and Discussion 162 

Notes and References 165 

Problems 166 


I CHAPTER 3 Pulse Modulation 183 


3.1 
3,2 
3.3 
3.4 
3.5 
3.6 
3.7 
3.8 
3.9 
3.10 
3.11 
3.12 
3.13 
3.14 
3.15 


Introduction 183 

Sampling Process 184 

Pulse-Amplitude Modulation 188 

Other Forms of Pulse Modulation 191 
Bandwidth-Noise Trade-off 193 

Quantization Process 193 

Pulse-Code Modulation 201 

Noise Considerations in PCM Systems 209 
Time-Division Multiplexing 211 

Digital Multiplexers 214 

Virtues, Limitations, and Modifications of PCM 217 
Delta Modulation 218 

Linear Prediction 223 

Differential Pulse-Code Modulation 227 
Adaptive Differential Pulse-Code Modulation 229 


CONTENTS xv 


3.16 Computer Experiment: Adaptive Delta Modulation 232 
3.17 MPEG Audio Coding Standard 234 
3.18 Summary and Discussion 236 

Notes and References 238 

Problems 239 


i CHAPTER 4 Baseband Pulse Transmission 247 


4,1 Introduction 247 
4.2 Matched Filter 248 
4.3 Error Rate Due to Noise 253 
4.4 Intersymbol Interference 259 
4.5 Nyquist’s Criterion for Distortionless Baseband Binary Transmission 261 
4.6 Correlative-Level Coding 267 
4.7 Baseband M-ary PAM Transmission 275, 
4.8 _ Digital Subscriber Lines 277 
4.9 Optimum Linear Receiver 282 
4.10 Adaptive Equalization 287 
4.11 Computer Experiments: Eye Patterns 293 
4.12 Summary and Discussion 296 
Notes and References 297 
Problems 300 


| CHAPTER 5 Signal-Space Analysis 309 


5.1 Introduction 309 
5.2 Geometric Representation of Signals 311 
5.3 Conversion of the Continuous AWGN Channel into a Vector Channel 318 
5.4 Likelihood Functions 322 
5.5 Coherent Detection of Signals in Noise: Maximum Likelihood Decoding 322 
5.6 Correlation Receiver 326 
5.7 Probability of Error 328 
5.8 Summary and Discussion 337 
Notes and References 337 
Problems 338 


|Cuaprer 6 Passband Digital Transmission 344 








6.1 Introduction 344 
6.2 Passband Transmission Model 348 
6.3 Coherent Phase-Shift Keying 349 


xvi CONTENTS 


6.4 
6.5 
6.6 
6.7 
6.8 
6.9 
6.10 
6.11 
6.12. 
6.13 
6.14 
6.15 
6.16 


Hybrid Amplitude/Phase Modulation Schemes 368 

Coherent Frequency-Shift Keying 380 

Detection of Signals with Unknown Phase 403 

Noncoherent Orthogonal Modulation 407 

Noncoherent Binary Frequency-Shift Keying 413 

Differential Phase-Shift Keying 414 

Comparison of Digital Modulation Schemes Using a Single Carrier 417 
Voiceband Modems 420 

Multichannel Modulation 431 

Discrete Multitone 440 

Synchronization 448 

Computer Experiments: Carrier Recovery and Symbol Timing 458 
Summary and Discussion 464 

Notes and References 465 

Problems 468 


] CHAPTER 7 Spread-Spectrum Modulation 479 


7.1 
7.2 
7.3 
7.4 
7.5 
7.6 
7.7 
7.8 
7.9 


Introduction 479 

Pseudo-Noise Sequences 480 

A Notion of Spread Spectrum 488 

Direct-Sequence Spread Spectrum with Coherent Binary Phase-Shift Keying 490 
Signal-Space Dimensionality and Processing Gain 493 
Probability of Error 497 

Frequency-Hop Spread Spectrum 499 

Computer Experiments: Maximal-Length and Gold Codes 505 
Summary and Discussion 508 

Notes and References 509 

Problems 509 


i CHAPTER 8 Multiuser Radio Communications 512 


8.1 
8.2 
8.3 
8.4 
8.5 
8.6 
8.7 
8.8 
8.9 


Introduction 512 

Multiple-Access Techniques 513 

Satellite Communications 514 

Radio Link Analysis 517 

Wireless Communications 529 | 

Statistical Characterization of Multipath Channels 535 
Binary Signaling over a Rayleigh Fading Channel 542 
TDMA and CDMA Wireless Communication Systems 547 
Source Coding of Speech for Wireless Communications 550 


CONTENTS xvii 





8.10 Adaptive Antenna Arrays for Wireless Communications 553 
8.11 Summary and Discussion 559 
Notes and References 560 
Problems 562 
| CHAPTER 9 Fundamental Limits in Information Theory 567 
9.1 Introduction 567 
9.2 Uncertainty, Information, and Entropy 568 
9.3 Source-Coding Theorem 574 
9.4 Data Compaction 575 
9.5 Discrete Memoryless Channels 581 
9.6 Mutual Information 584 
9.7 Channel Capacity 587 
9.8 Channel-Coding Theorem 589 i 
9.9 Differential Entropy and Mutual Information for Continuous Ensembles 593 
9,10 Information Capacity Theorem 597 
9.11 Implications of the Information Capacity Theorem 601 
9.12 Information Capacity of Colored Noise Channel 607 
9,13 Rate Distortion Theory 611 : 
9.14 Data Compression 614 
9,15 Summary and Discussion 616 


Notes and References 617 
Problems 618 


| Cuaprer 10 Error-Control Coding 626 


10.1 
10.2 
10.3 
10.4 
10.5 
10.6 
10.7 
10.8 
10.9 
10.10 
10.11 
10.12 


Introduction 626 

Discrete-Memoryless Channels 629 

Linear Block Codes 632 

Cyclic Codes 641 

Convolutional Codes 654 

Maximum Likelihood Decoding of Convolutional Codes, 660 
Trellis-Coded Modulation 668 i 
Turbo Codes 674 

Computer Experiment: Turbo Decoding 682 
Low-Density Parity-Check Codes 683 

Irregular Codes 691 

Summary and Discussion 693 

Notes and References 694 

Problems 696 


xviii CONTENTS 


APPENDIX 1 Probability Theory 703 

APPENDIX 2 Representation of Signals and Systems 715 

APPENDIX 3 Bessel Functions 735 

APPENDIX 4 Confluent Hypergeometric Functions 740 

APPENDIX 5 Cryptography 742 

APPENDIX 6 Tables 761 

GLOSSARY 77% 
BIBLIOGRAPHY 777 


INDEX 792 





BACKGROUND 
AND PREVIEW 


The background and preview material presented herein sets the stage for a statistical 
treatment of communication systems in subsequent chapters. In particular, we describe the 
following: 


> The communication process. 

> Primary communication resources, namely, transmitted power and channel bandwidth. 
> Sources of information. 

> The two primary types of switching: circuit switching and packet switching. 


> Communication channels for the transportation of information-bearing signals from the 
transmitter to the receiver. 


> The modulation process, which is basic to communication systems. 
> Analog and digital types of communication systems. 
> Shannon’s information capacity theorem. 


> A digital communications problem. 


The chapter concludes with some historical notes, as a source of motivation for the 
reader. 


i The Communication Process 


Today, communication enters our daily lives in so many different ways that it is very easy 
to overlook the multitude of its facets. The telephones at our hands, the radios and tele- 
visions in our living rooms, the computer terminals with access to the Internet in our offices 
and homes, and our newspapers are all capable of providing rapid communications from 
every corner of the globe. Communication provides the senses for ships on the high seas, 
aircraft in flight, and rockets and satellites in space. Communication through a wireless 
telephone keeps a car driver in touch with the office or home miles away. Communication 
keeps a weather forecaster informed of conditions measured by a multitude of sensors. 
Indeed, the list of applications involving the use of communication in one way or another 
is almost endless. 


2 m BACKGROUND AND PREVIEW 


In the most fundamental sense, communication involves implicitly the transmission 
of information from one point to another through a succession of processes, as described 
here: 


1. The generation of a message signal: voice, music, picture, or computer data. - 

2. The description of that message signal with a certain measure of precision, by a set 
of symbols: electrical, aural, or visual. 

3. The encoding of these symbols in a form that is suitable for transmission over a 
physical medium of interest. 

4. The transmission of the encoded symbols to the desired destination. 

5. The decoding and reproduction of the original symbols. 

6. The re-creation of the original message signal, with a definable degradation in qual- 
ity; the degradation is caused by imperfections in the system. 


There are, of course, many other forms of communication that do not directly involve 
the human mind in real time. For example, in computer communications involving com- 
munication between two or more computers, human decisions may enter only in setting 
up the programs or commands for the computer, or in monitoring the results. 

Irrespective of the form of communication process being considered, there are three 
basic elements to every communication system, namely, transmitter, channel, and receiver, 
as depicted in Figure 1. The transmitter is located at one point in space, the receiver is 
located at some other point separate from the transmitter, and the channel is the physical 
medium that connects them. The purpose of the transmitter is to convert the message signal 
produced by the source of information into a form suitable for transmission over the 
channel. However, as the transmitted signal propagates along the channel, it is distorted 
due to channel imperfections. Moreover, noise and interfering signals (originating from 
other sources) are added to the channel output, with the result that the received signal is 
a corrupted version of the transmitted signal. The receiver has the task of operating on 
the received signal so as to reconstruct a recognizable form of the original message signal 
for a user. 

There are two basic modes of communication: 


1. Broadcasting, which involves the use of a single powerful transmitter and numerous 
receivers that are relatively inexpensive to build. Here information-bearing signals 
flow only in one direction. — 

2. Point-to-point communication, in which the communication process takes place over 
a link between a single transmitter and a receiver. In this case, there is usually a 
bidirectional flow of information-bearing signals, which requires the use of a trans- 
mitter and receiver at each end of the link. 


Communication System 
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FIGURE Í Elements of a communication system. 


Sources of Information 3 


The broadcasting mode of communication is exemplified by radio and television, and the 
ubiquitous telephone provides the means for one form of point-to-point communication. 
Another example of point-to-point communication is the link between an Earth station 
and a robot navigating the surface of a distant planet. 

All these different communication systems as well as others not mentioned here share 
a common feature: The underlying communication process in each and every one of them 
is statistical in nature. Indeed, it is for this important reason that much of this book is 
devoted to the statistical underpinnings of communication systems. In so doing, we develop 
an exposition of the fundamental issues involved in the study of different communication 
methodologies and thereby provide a natural forum for their comparative evaluations. 


i Primary Communication Resources 





In a communication system, two primary resources are employed: transmitted power and 
channel bandwidth. The transmitted power is the average power of the transmitted signal. 
The channel bandwidth is defined as the band of frequencies allocated for the transmission 
of the message signal. A general system design objective is to use these two resources as 
efficiently as possible. In most communication channels, one resource may be considered 
more important than the other. We may therefore classify communication channels as 
power limited or band limited. For example, the telephone circuit is a typical band-limited 
channel, whereas a space communication link or satellite channel is typically power 
limited. 

When the spectrum of a message signal extends down to zero or low frequencies, we 
define the bandwidth of the signal as that upper frequency above which the spectral content 
of the signal is negligible and therefore unnecessary for transmitting information. For 
example, the average voice spectrum extends well beyond 10 kHz, though most of the 
average power is concentrated in the range of 100 to 600 Hz, and a band from 300 to 
3100 Hz gives good articulation. Accordingly, we find that telephone circuits that respond 
well to this latter range of frequencies give quite satisfactory commercial telephone service. 

Another: important point that we have to keep in mind is the unavoidable presence 
of noise in a communication system. Noise refers to unwanted waves that tend to disturb 
the transmission and processing of message signals in a communication system. The 
sources of noise may be internal or external to the system. 

A quantitative way to account for the effect of noise is to introduce signal-to-noise 
ratio (SNR) as a system parameter. For example, we may define the SNR at the receiver 
input as the ratio of the average signal power to the average noise power, both being 
measured at the same point. The customary practice is to express the SNR in decibels 
(dBs), defined as 10 times the logarithm (to base 10) of the power ratio. For example, 
signal-to-noise ratios of 10, 100, and 1,000 correspond to 10, 20, and 30 dBs, respectively. 


i Sources of Information 


The telecommunications environment is dominated by four important sources of infor- 
mation: speech, music, pictures, and computer data. A source of information may be 
characterized in terms of the signal that carries the information. A signal is defined as a 
single-valued function of time that plays the role of the independent variable; at every 
instant of time, the function has a unique value. The signal can be one-dimensional, as in 
the case of speech, music, or computer data; two-dimensional, as in the case of pictures; 
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three-dimensional, as in the case of video data; and four-dimensional, as in the case of 
volume data over time. In the sequel, we elaborate on different sources of information. 


(i) 


Speech is the primary method of human communication. Specifically, the speech 
communication process involves the transfer of information from a speaker to a 
listener, which takes place in three successive stages: 


> Production. An intended message in the speaker’s mind is represented by a speech 
signal that consists of sounds (i.e., pressure waves) generated inside the vocal tract 
and whose arrangement is governed by the rules of language. 


> Propagation. The sound waves propagate through the air at a speed of 300 m/s, 
reaching the listener’s ears, 

» Perception, The incoming sounds are deciphered by the listener into a received 
message, thereby completing the chain of events that culminate in the transfer of 
information from the speaker to the listener. 


The speech-production process may be viewed as a form of filtering, in which a sound 
source excites a vocal tract filter. The vocal tract consists of a tube of nonuniform 
cross-sectional area, beginning at the glottis (i.e., the opening between the vocal 
cords) and ending at the lip. As the sound propagates along the vocal tract, the 
spectrum (i.e., frequency content) is shaped by the frequency selectivity of the vocal 
tract; this effect is somewhat similar to the resonance phenomenon observed in organ 
pipes. The important point to note here is that the power spectrum (i.e., the distri- 
bution of long-term average power versus frequency) of speech approaches zero for 
zero frequency and reaches a peak in the neighborhood of a few hundred hertz. To 
put matters into proper perspective, however, we have to keep in mind that the 
hearing mechanism is very sensitive to frequency. Moreover, the type of communi- 
cation system being considered has an important bearing on the band of frequencies 
considered to be “essential” for the communication process. For example, as men- 
tioned previously, a bandwidth of 300 to 3100 Hz is considered adequate for com- 
mercial telephonic communication. 


The second source of information, music, originates from instruments such as the 
piano, violin, and flute. The note made by a musical instrument may last for a short 
time interval as in the pressing of a key on a piano, or it may be sustained for a long 
time interval as in the example of a flute player holding a prolonged note. Typically, 
music has two structures: a melodic structure consisting of a time sequence of sounds, 
and a harmonic structure consisting of a set of simultaneous sounds. Like a speech 
signal, a musical signal is bipolar. However, a musical signal differs from a speech 
signal in that its spectrum occupies a much wider band of frequencies that may extend 
up to about 15 kHz. Accordingly, musical signals demand a much wider channel 
bandwidth than speech signals for their transmission. 


The third source of information, pictures, relies on the human visual system for its 
perception. The picture can be dynamic, as in television, or static, as in facsimile. 
Taking the case of television first, the pictures in motion are converted into electrical 
signals to facilitate their transport from the transmitter to the receiver. To do so, 
each complete picture is sequentially scanned. The scanning process is carried out in 
a TV camera. In a black-and-white TV, the camera contains optics designed to focus 
an image on a photocathode consisting of a large number of photosensitive elements. 
The charge pattern so generated on the photosensitive surface is scanned by an elec- 
tron beam, thereby producing an output current that varies temporally with the way 
in which the brightness of the original picture varies spatially from one point to 
another. The resulting output current is called a video signal. The type of scanning 
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used in television is a form of spatial sampling called raster scanning, which converts 
a two-dimensional image intensity into a one-dimensional waveform; it is somewhat 
analogous to the manner in which we read a printed paper in that the scanning is 
performed from left to right on a line-by-line basis. In North American analog tele- 
vision, a picture is divided into 525 lines, which constitute a frame. Each frame is 
decomposed into. two interlaced fields, each of which consists of 262.5 lines. For 
convenience of presentation, we will refer to the two fields as I and II. The scanning 
procedure is illustrated in Figure 2. The lines of field I are depicted as solid lines, and 
those of field II are depicted as dashed lines. The start and end of each field are also 
included in the figure. Field I is scanned first. The scanning spot of the TV camera 
moves with constant velocity across each line of the field from left to right, and the 
image intensity at the center of the spot is measured; the scanning spot itself is partly 
responsible for local spatial averaging of the image. When the end of a particular 
line is reached, the scanning spot quickly flies back (in a horizontal direction) to the 
start of the next line down in the field. This flyback is called the horizontal retrace. 
The scanning process described here is continued until the whole field has been ac- 
counted for. When this condition is reached, the scanning spot moves quickly (in a 
vertical direction) from the end of field I to the start of field II. This second flyback 
is called the vertical retrace. Field II is treated in the same fashion as field I. The time 
taken for each field to be scanned is 1/60 s. Correspondingly, the time taken for a 
frame or a complete picture to be scanned is 1/30 s. With 525 lines in a frame, the 
line-scanning frequency equals 15.75 kHz. Thus, by flashing 30 still pictures per 
second on the display tube of the TV receiver, the human eye perceives them to be 
moving pictures. This effect is due to a phenomenon known as the persistence of 
vision. During the horizontal- and vertical-retrace intervals, the picture tube is made 
inoperative by means of blanking pulses that are generated at the transmitter. More- 
over, synchronization between the various scanning operations at both transmitter 
and receiver is accomplished by means of special pulses that are transmitted during 
the blanking periods; thus, the synchronizing pulses do not show on the reproduced 
picture. The reproduction quality of a TV picture is limited by. two basic factors: 

1. The number of lines available in a raster scan, which limits resolution of the 

picture in the vertical direction. 
2. The channel bandwidth available for transmitting the video signal, which limits 
resolution of the picture in the horizontal direction. 
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FIGURE 2 Interlaced raster scan. 
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For each direction, resolution is expressed in terms of the maximum number of lines 
alternating between black and white that can be resolved in the TV image along the 
pertinent direction by a human observer. In the NTSC (National Television System 
Committee) system, which is the North American standard, the parameter values 
used result in a video bandwidth of 4.2 MHz, which extends down to zero frequency. 
This bandwidth is orders of magnitude larger than that of a speech signal. Note also 
that whereas a speech signal is bipolar, a video (television) signal is inherently positive 
(i.e., unipolar). 

In color TV, the perception of color is based on the three types of color recep- 
tors (cones) in the human eye: red, green, and blue, whose wavelengths are 570 nm, 
535 nm, and 445 nm, respectively. These three colors are referred to as primary 
colors because any other color found in nature can be approximated by an additive 
mixture of them. This physical reality is indeed the basis for the transmission of color 
in commercial TV broadcasting. The three primary colors are represented by the 
video signals mt,(t), #t¢(t), and m,(t), respectively. To conserve bandwidth and pro- 
duce a picture that can be viewed on a conventional black-and-white (monochrome) 
television receiver, the transmission of these three primary colors is accomplished by 
observing that they can be uniquely represented by any three signals that are inde- 
pendent linear combinations of z(t), m¢(t), and m,(t). The three signals are as 
follows: 


> A luminance signal, m;,(t), which produces a black-and-white version of the color 
picture when it is received on a conventional monochrome television receiver. 

» A pair of signals, m;(t) and mo(t), called the chrominance signals, which indicate 
the way the color of the picture departs from shades of gray. 


The luminance signal m,(t) is assigned the entire 4.2 MHz bandwidth. Owing to 
certain properties of human vision, tests show that if the nominal bandwidths of the 
chrominance signals m(t) and mg(t) are 1.6 MHz and 0.6 MHz, respectively, sat- 
isfactory color reproduction is possible. 

Turning next to a facsimile (fax) machine, the purpose of this machine is to 
transmit still pictures over a communication channel (most notably, a telephone 
channel), Such a machine provides a highly popular facility for the transmission of . 
handwritten or printed text from one point to another; transmitting text by facsimile 
is treated simply like transmitting a picture. The basic principle employed for signal 
generation in a facsimile machine is to scan an original document (picture) and use 
an image sensor to convert the light to an electrical signal. 

Finally, personal computers (PCs) have become an integral part of our daily lives. 
We use them for electronic mail, exchange of software, and sharing of resources. The 
text transmitted by a PC is usually encoded using the American Standard Code for 
Information Interchange (ASCII), which is the first code developed specifically for 
computer communications. Each character in ASCII is represented by seven data bits 
constituting a unique binary pattern made up of Os and 1s; bit is acronym for binary 
digit. Thus a total of 27 = 128 different characters can be represented in ASCII. The 
characters are various lowercase and uppercase letters, numbers, special control sym- 
bols, and punctuation symbols commonly used such as @, $, and %. Some of the 
special “control” symbols, such as BS (backspace) and CR (carriage return), are used 
to control the printing of characters on a page. Other symbols, such as ENQ (enquiry) 
and ETB (end of transmission block), are used for communication purposes. (A com- 
plete listing of ASCII characters is given in Table A6.1.) The seven data bits are 
ordered starting with the most significant bit b down to the least significant bit b1, 
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FIGURE 3 The bit format for sending asynchronous serial data used in the RS-232 
standard. 


as illustrated in Figure 3. At the end of the data bits, an extra bit bg is appended for 
the purpose of error detection. This error-detection bit is called a parity bit. A se- 
quence of eight bits is referred to as a-byte, or an octet. The parity bit is set in such 
a way that the total number of 1s in each byte is odd for odd parity and even for 
even parity. Suppose, for example, the communicators agree to use even parity; then 
the parity bit will be a 0 when the number of 1s in the data bit is even and a 1 when 
it is odd. Hence, if a single bit in a byte is received in error and thereby violates the 
even parity rule, it can be detected and then corrected through retransmission. Per- 
sonal computers are often connected via their RS (recommended standard)-232 ports. 
When ASCII data (in fact, all character data) are transmitted through these ports, a 
start bit, set to 0, and one or more stop bits, set to 1, as shown in Figure 3, are added 
to provide character framing. When the transmission is idle, a long series of 1s is 
sent so as to keep the circuit connection alive. In Figure 3, symbols 0 and 1 are 
designated as “low” and “high,” respectively. They are also sometimes referred to 

s “space” and “mark,” respectively; the latter terminology comes from the days of 
telegraphy. The text prepared on a PC is usually stored and then transmitted over a 
communication channel (e.g., a telephone channel) with a single character being sent 
at a time. This form of data transmission is called asynchronous transmission, as 
opposed to synchronous transmission in which a whole sequence of encoded char- 
acters is sent over the channel in one long transmission. Encoded characters produced 
by a mixture of asynchronous and synchronous terminals are combined by means 
of data multiplexers. The multiplexed stream of data so formed is then applied to a 
device called a modem (modulator-demodulator) for the purpose of transmission 
over the channel. 

In summary, computer-generated data and television signals are both wide- 
band signals, in that their power content occupies a wide range of frequencies. An- 
other important characteristic of data communication between personal computers 
is burstiness, which means that information is usually transmitted from one terminal 
to another in bursts with silent periods between bursts. Indeed, data traffic involving 
computers in one form or another tends to be of a bursty nature. This is to be 
contrasted with traffic in a digital transmission network due to voice or interactive 
video, which, relatively speaking, is continuous. 

Another way in which we use the computer is to download compressed forms 
of text, audio, and video data from a service provider at a remote location. Data 
compression provides a practical means for the efficient storage and transmission of 
these kinds of data. A data compression system consists of an encoder and a decoder, 
where the compression of an incoming data stream and its reconstruction are per- 
formed, respectively. Basically, there are two forms of data compression: 


1. Lossless compression operates by removing the redundant information contained 
in the data of interest. The compression is said to be lossless because it is com- 
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pletely reversible in that the original data can be reconstructed exactly. Lossless 
compression is also referred to as data compaction. 

2. Lossy compression involves the loss of information in a controlled manner; the 
compression may therefore not be completely reversible. Lossy compression is, 
however, capable of achieving a compression ratio higher than that attainable 
with lossless methods. 

For digital text, lossless compression is required. In this context, we mention the 

Lempel-Ziv algorithm, which is intrinsically adaptive and capable of encoding 

groups of source symbols that occur frequently. It achieves a compression of ap- 

proximately 55 percent on ordinary English text, which, loosely speaking, corre- 
sponds to the compression that would be achieved by encoding pairs of letters. The 

Lempel-Ziv algorithm is a form of entropic coding, or source coding, which is dis- 

cussed in Chapter 9. 

In many other applications, lossy compression is usually the preferred approach 
as its use can substantially reduce the data size without significantly altering the 
perceptual quality of an image or audio signal. For such applications, this form of 
data compression is acceptable, and in high-throughput data-transmission applica- 
tions such as the Internet, it is a necessity. But in some other applications such as a 
clinical setting, the quality of a medical image (e.g., digital x-ray radiograph) must 
not be degraded on reconstruction. 

For digital audio and video applications involving storage or transmission to 
be viable in today’s marketplace, we need standard compression algorithms that 
enable the interoperability of equipment produced by different manufacturers. In this 
context, we mention three prominent standard compression algorithms that cater to 
different needs: 
> The JPEG image coding standard? is designed to compress full-color or grayscale 

images of natural, real-world scenes by exploiting known limitations of the human 
visual system; JPEG stands for Joint Photographic Experts Group. At the input to 
the encoder, picture elements, or pixels, are grouped into 8 X 8 blocks, which are 
applied to a relative of the Fourier transform known as the discrete cosine trans- 
form (DCT). The DCT decomposes each block of pixels into a set of 64 coeffi- 
cients that closely satisfy two related objectives: 

1. The coefficients should be as uncorrelated as possible. 

2. The energy of the input signal should be packed into the smallest number of 

coefficients possible. 

The next operation in the encoder is that of quantization, where each of the 64 
DCT coefficients is rounded off. In JPEG, quantization is performed in conjunction 
with a quantization table supplied by the user as an input to the encoder. Each 
element of the table is an integer from 1 to 255 that specifies the step size of the 
DCT coefficients, which, in turn, permits the representation of each quantized 
DCT coefficient by an 8-bit code word. Basically, the purpose of quantization is 
to discard information that is not perceptually discernible. Quantization is a many- 
to-one mapping and therefore the principal source of lossiness in the encoder. The 
final operation in the encoder is that of Huffman coding, which is a form of 
entropic (source) coding also discussed in Chapter 9. Huffman coding achieves 
additional data compression in a lossless manner by encoding the quantized DCT 
coefficients in accordance with their statistical characteristics. At the decoder, data 
reconstruction is performed through a sequence of operations that are the inverse 
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of those-in the encoder, namely, Huffman decoding, dequantization in accordance 

with the quantization table, and finally the inverse DCT. 

The MPEG-1/video coding standard’ is designed primarily to compress video sig- 

nals at 30 frames per second (fps) into bit streams running at the rate of 1.5 

megabits per second (Mb/s); MPEG stands for Motion Photographic Experts 

Group. The MPEG-1 video coding standard achieves this design goal by exploiting 

four basic forms of redundancy inherently present in video data: 

1. Interframe (temporal) redundancy. 

2. Interpixel redundancy within a frame. 

3. Psychovisual redundancy. 

4. Entropic coding redundancy. 

It is the exploitation of interframe redundancy that distinguishes MPEG-1 from 

JPEG. In principle, neighboring frames in typical video sequences are highly cor- 

related. The meaning of this high correlation is that, in an average sense, a video 

signal does not change rapidly from one frame to the next, and as a result, the 
difference between adjacent frames has a variance (i.e., average power) that is 
much smaller than the variance of the video signal itself. Accordingly, the inter- 
frame redundancy can be significantly reduced to produce a more efficiently com- 
pressed video signal. This reduction is achieved through the use of prediction to 
estimate each frame from its neighbors; the resulting prediction error is transmitted 
for motion estimation and compensation. The prediction is nonlinear by virtue of 
the nature of the problem. As with-JPEG, the interpixel redundancy is reduced 
through the combined use of the DCT, quantization, and lossless entropic coding. 

The net result is that full-motion.video becomes a 1.5 Mb/s stream of computer 

data that can be stored on compact discs or integrated with texts and graphics. 

Most important, the full-motion video and associated audio can be delivered over 

existing computer and telecommunication networks, which, in turn, makes it pos- 

sible to fulfill the need for video-on-demand on the Internet. 

The MPEG-1/audio coding standard? is based on perceptual coding, which is a 

waveform-preserving process; that is, the amplitude-time waveform of the decoded 

audio signal closely approximates that of the original audio signal. In basic terms, 
the encoding process encompasses four distinct operations: 

1. Time-frequency mapping, whereby the input audio signal is decomposed into 
multiple subbands. 

2. Psychoacoustic modeling, which simultaneously operates on the input audio 
signal to compute certain thresholds using known rules from the psychoacous- 
tic behavior of the human auditory system. 

3. Quantization and coding, which, in conjunction with the psychoacoustic 
model, works on the output of the time-frequency mapper so as to maintain 
the noise resulting from quantization process at an inaudible level. 

4. Frame-packing, which is used to format the quantized audio samples into a 
decodable bit stream. 

The psychoacoustic model builds on a perceptual phenomenon known as auditory 

masking. Specifically, the human ear does not perceive quantization noise in a 

given frequency band if the average noise power lies below the masking threshold 

(i.e., the threshold of just noticeable distortion). For a given frequency band of 

interest, the masking threshold varies with frequency across that band. The min- 
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imum masking threshold is the one that is employed in the psychoacoustic model 
on a band-by-band basis. For example, the net result of using the MPEG-1 stan- 
dard on the two audio channels of a stereo program is that each digitized audio 
signal, coming in at the rate of 768 kilobits per second (kb/s), is compressed to a 
rate as low as 16 kb/s. (The incoming data rate of 768 kb/s corresponds to a 
sampling rate of 48 kHz, with each sample being represented by a 16-bit code 
word.) Thus the MPEG-1/audio coding standard is suitable for the storage of 
audio signals in inexpensive media or their transmission over channels with limited 
bandwidth, while at the same time maintaining perceptual quality. 


i Communication Networks® 


A communication network (or simply network), illustrated in Figure 4, consists of an 
interconnection of a number of routers made up of intelligent processors (e.g., microproc- 
essors). The primary purpose of these processors is to route data through the network, 
hence the name. Each router has one or more hosts attached to it; hosts are devices that 
communicate with one another. The network is designed to serve as a shared resource for 
moving data exchanged between hosts in an efficient manner and to provide a framework 
to support new applications and services. 

The telephone network is an example of a communication network in which circuit 
switching is used to provide a dedicated communication path, or circuit, between two 
hosts. The circuit consists of a connected sequence of links from source to destination. For 
example, the links may consist of time slots for which a common channel is available for 
access by a multitude of users. The circuit, once in place, remains uninterrupted for the 
duration of transmission. Circuit switching is usually controlled by a centralized hierar- 
chical control mechanism with knowledge of the network’s organization. To establish a 
circuit-switched connection, an available path through the network is seized and then 
dedicated to the exclusive use of the two hosts wishing to communicate. In particular, a 
call-request signal must propagate all the way to the destination and be acknowledged 
before transmission can begin. Then, the network is effectively transparent to the users. 
This means that during the connection time, the bandwidth and resources allocated to the 
circuit are essentially “owned” by the two hosts until the circuit is disconnected. The circuit 
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FIGURE 4 Communication network. 
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thus represents an efficient use of resources only to the extent that the allocated bandwidth 
is properly used. Although the telephone network is used to transmit data, voice constitutes 
the bulk of the network’s traffic. Indeed, circuit switching is well suited to the transmission 
of voice signals, since voice gives rise to a stream traffic and voice conversations tend to 
be of long duration (about 2 minutes on the average) compared to the time required for 
setting up the circuit (about 0.1 to 0.5 seconds). Moreover, in most voice conversations, 
there is information flow for a relatively large percentage of the connection time, which 
makes circuit switching all the more suitable for voice conversations. 

In circuit switching, a communication link is shared between the different sessions 
using that link on a fixed allocation basis. In packet switching, on the other hand, the 
sharing is done on a demand basis, so it has an advantage over circuit switching in that 
when a link has traffic to send, the link may be more fully utilized. 

The network principle of packet switching is “store and forward.” Specifically, in a 
packet-switched network, any message larger.than a specified size is subdivided prior to 
transmission into segments not exceeding the specified size. The segments are commonly 
referred to as packets. The original message is reassembled at the destination on a packet- 
by-packet basis. The network may be viewed as a distributed pool of network resources 
(i.e., channel bandwidth, buffers, and switching processors) whose capacity is shared dy- 
namically by a community of competing hosts wishing to communicate. In contrast, in a 
circuit-switched network, resources are dedicated tó a pair of hosts for the entire period 
they are in session. Accordingly, packet switching is far better suited to a computer- 
communication environment in which bursts of data are exchanged between hosts on an 
occasional basis. The use of packet switching, however, requires that careful control be 
exercised on user demands; otherwise, the network may be seriously abused. 

The design of a data network (i.e., a network in which the hosts are all made up of 
computers and terminals) may proceed in an orderly way by looking at the network in 
terms of a layered architecture, regarded as a hierarchy of nested layers. Layer refers to a 
process or device inside a computer system, designed to perform a specific function. Nat- 
urally, the designers of a layer will be intimately familiar with its internal details and 
operation. At the system level, however, a user views the layer merely as a “black box” 
that is described in terms of inputs, outputs, and the functional relation between outputs 
and inputs. In a layered architecture, each layer regards the next lower layer as one or 
more black boxes with some given functional specification to be used by the given higher 
layer. Thus, the highly complex communication problem in data networks is resolved as 
a manageable set of well-defined interlocking functions. It is this line of reasoning that has 
led to the development of the open systems interconnection (OSI)’ reference model by a 
subcommittee of the International Organization for Standardization. The term open refers 
to the ability of any two systems conforming to the reference model and its associated 
standards to interconnect. 

In the OSI reference model, the communications and related-connection functions 
are organized as a series of layers, or levels, with well-defined interfaces, and with each 
layer built on its predecessor. In particular, each layer performs a related subset of primitive 
functions, and it relies on the next lower layer to perform additional primitive functions. 
Moreover, each layer offers certain services to the next higher layer and shields the latter 
from the implementation details of those services. Between each pair of layers, there is an 
interface. It is the interface that defines the services offered by the lower layer to the upper 
layer. 

The OSI model is composed of seven layers, as illustrated in Figure 5; this figure also 
includes a description of the functions of the individual layers of the model. Layer k on 
system A, say, communicates with layer k on some other system B in accordance with a 
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set of rules and conventions, collectively constituting the layer k protocol, where k = 1, 
2,...7. (The term protocol has been borrowed from common usage, describing conven- 
tional social behavior between human beings.) The entities that comprise the correspond- 
ing layers on different systems are referred: to as peer processes. In other words, commu- 
nication is achieved by having the peer processes in two different systems communicate 
via a protocol, with the protocol itself being defined by a set of rules of procedure. Physical 
communication between peer processes exists only at layer 1. On the other hand, layers 2 
through 7 are in virtual communication with their distant peers. However, each of these 
six layers can exchange data and control information with its neighboring layers (below 
and above) through layer-to-layer interfaces. In Figure 5, physical communication is shown 
by solid lines and virtual communication by dashed lines. 
Our primary interest in this book is in the physical layer of the OSI model. 


= INTERNET 


Any discussion of computer networks naturally leads to the Internet. In the Internet par- 
adigm, the underlying network technology is decoupled from the applications at hand by 
adopting an abstract definition of network service. In more specific terms, we may say the 
following: 


» The applications are carried out independently of the technology employed to con- 
struct the network. 

> By the same token, the network technology is capable of evolving without affecting 
the applications. 


The Internet architecture, depicted in Figure 6, has three functional blocks: hosts, 
subnets, and routers. The hosts constitute nodes of the network, where data originate or 
where they are delivered. The routers constitute intermediate nodes that are used to cross 
subnet boundaries. Within a subnet, all the hosts belonging to that subnet exchange data 
directly; see, for example, subnets 1 and 3 in Figure 6. 

Like other computer networks, the Internet has a layered set of protocols. In partic- 
ular, the exchange of data between the hosts and routers is accomplished by means of the 
Internet protocol (IP), as illustrated in Figure 7. The IP is a universal protocol that resides 
in the network layer (i.e., layer 3 of the OSI reference model). It is simple, defining an 
addressing plan with a built-in capability to transport data in the form of packets from 
node to node. In crossing a subnetwork boundary, the routers make the decisions as to 
how the packets addressed for a specified destination should be routed. This is done on 
the basis of routing tables that are developed through the use of custom protocols for 








Hosts Hosts 


FIGURE 6 An interconnected network of subnets. 
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FIGURE 7 Illustrating the network architecture of the Internet. 


exchanging pertinent information with other routers. The net result of using the layered 
set of protocols is the delivery of best effort service. That is, the Internet offers to deliver 
each packet of data, but there are no guarantees on the transit time experienced in delivery 
or even whether the packets will be delivered to the intended recipient. 


& BROADBAND NETWORKS 


With the ever-increasing demand for new services (e.g., video on demand, multimedia 
communications) and the availability of key enabling technologies (e.g., optical fibers, 
digital switches), the telephone network is evolving into an all-purpose broadband network 
known as the broadband integrated services digital network (B-ISDN). The underlying 
technology that makes B-ISDN possible is a user-network interface protocol called the 
asynchronous transfer mode (ATM). ATM is a high-bandwidth, low-delay, packet-like 
technique used for switching and multiplexing; it is independent of the physical means of 
transport. The low-delay feature of the technique is needed to support real-time services 
such as voice. The high-bandwidth feature is required to handle video on demand. Simply 
put, ATM is both a technology that is hidden from the users and a connection-oriented 
service that is visible to the users. 

As the name implies, ATM is not synchronous (i.e., tied to a master clock). It allows 
for the transport of digital information in the form of small, fixed-size packets called cells. 
The key feature of ATM to note here is that the connection-oriented service preserves call 
sequencing, which means that no reassembly of cells is needed prior to presenting the traffic 
stream to the destination host. The deployment of a cell-switching technology in B-ISDN 
is a gigantic break with the traditional use of circuit switching in the telephone network. 

The primary purpose of ATM is to allocate network resources (i.e., bandwidth, buf- 
fers, and processing horsepower) efficiently so as to guarantee the expected quality of 
service (QoS) for each connection. QoS is measured in terms of three parameters: 


® Cell loss ratio, defined as the ratio of the number of cells lost in transport across the 
network to the total number of cells pumped into the network. 

» Cell delay, defined as the time taken for a cell of a particular connection to transit 
across the network. 

> Cell delay variation, defined as the dispersion or jitter about the mean cell delay. 


Quality of service offered in B-ISDN is to be contrasted with best effort service offered by 
the Internet. 
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TARE I Hierarchy 
of SONET data rates 





Level Data Rate (Mb/s) 
OC-1 51.84 
OC-3 155.52 
Oc-9 466.56 
OC-12 622.08 
OC-18 933.12 
OC-24 1,244.16 
OC-36 1,866.24 


OC-48 2,488.32 


ĉOC stands for optical carrier level. 


After their generation, the ATM cells are structured for transport across the network. 
The cells in B-ISDN are placed on an optical transmission system called the synchronous 
optical network (SONET);® optical fibers are discussed in the next section. SONET uses 
time-division multiplexing, whereby the entire bandwidth of an optical fiber is devoted to 
different incoming data streams on a time-shared basis, hence the need for a synchronous 
operation. SONET is controlled by a master clock with an accuracy of about 1 part in 
10°. Thus bits of data are sent on a SONET line at extremely precise intervals, controlled 
by the master clock. Nevertheless, SONET permits the irregular time arrivals of ATM 
cells. l 

The basic SONFT frame is a block of 810 bytes put out every 125 us for an overall 
data rate of 51.84 Mb/s. Having 8000 frames every second exactly matches the sampling 
rate of 8 kHz, which is the standard sampling rate for the digital transmission of voice 
signals across the telephone network. The basic data rates of 51.84 Mb/s are synchronously 
byte-interleaved to generate a hierarchy of data rates, as summarized in Table 1. 


i Communication Channels 


The transmission of information across a communication network is accomplished in the 
physical layer by means of a communication channel. Depending on the mode of trans- 
mission used, we may distinguish two basic groups of communication channels: channels 
based on guided propagation and those based on free propagation. The first group includes 
telephone channels, coaxial cables, and optical fibers. The second group includes wireless 
broadcast channels, mobile radio channels, and satellite channels. These six channels are 
described in what follows. 


(i) As mentioned earlier, a typical telephone network uses circuit switching to establish 
an end-to-end communication link on a temporary basis. The primary purpose of 
the network is to ensure that the telephone transmission between a speaker at one 
end of the link and a listener at the other end is an acceptable substitute for 
face-to-face conversation. In this form of communication, the message source is the 
sound produced by the speaker’s voice, and the ultimate destination is the listener’s 
ear. The telephone channel, however, supports only the transmission of electrical 
signals. Accordingly, appropriate transducers are used at the transmitting and re- 
ceiving ends of the system. Specifically, a microphone is placed near the speaker’s 
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mouth to convert sound waves into an electrical signal, and the electrical signal is 
converted back into acoustic form by means of a mtoving-coil receiver placed near 
the listener’s ear. Present-day designs of these two transducers have been perfected 
so as to respond well to frequencies ranging from 20 to 8000 Hz; moreover, a pair 
of them can be compactly packaged inside a single telephone set that is easy to speak 
into or listen from. The telephone channel is a bandwidth-limited channel. The re- 
striction on bandwidth arises from the requirement of sharing the channel among a 
multitude of users at any one time. A practical solution to the telephonic commu- 
nication problem must therefore minimize the channel bandwidth requirement, sub- 
ject to a satisfactory transmission of human voice. To meet this requirement, the 
transducers and channel specifications must conform to standards based on subjec- 
tive tests that are performed on the intelligibility, or articulation, of telephone signals 
by representative. male and female speakers. A speech signal (male or female) is es- 
sentially limited to a band from 300 to 3100 Hz in the sense that frequencies outside 
this band do not contribute much to articulation efficiency. This frequency band may 
therefore be viewed as a rough guideline for the passband of a telephone channel 
that provides a satisfactory service, as illustrated in Figure 8 for a typical toll con- 
nection. Figure 8a shows the insertion loss of the channel plotted versus frequency; 
insertion loss (in dB) is defined as 10 logio(Po/P,), where P; is the power delivered 
to a load from a source via the channel and Py is the power delivered to the same 
load when it is connected directly to the source. Figure 8b shows the corresponding 
plot of the envelope (group) delay (in milliseconds) versus frequency; envelope delay 
is defined as the negative of the derivative of the phase response with respect to the 
angular frequency w = 2f. The plots of Figure 8 clearly illustrate the dispersive 
nature of the telephone channel. 

The telephone channel is built using twisted pairs for signal transmission. A 
twisted pair consists of two solid copper conductors, each of which is encased in a 
polyvinylchloride (PVC) sheath. Typically, each pair has a twist rate of 2 to 12 twists 
per foot, and a characteristic impedance of 90 to 110 ohms. Twisted pairs are usually 
made up into cables, with each cable consisting of many pairs in close proximity to 
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Ficune 8 Characteristics of typical telephone connection: (a) Insertion loss. (b) Envelope delay. 
(Adapted from Bellamy, 1991.) 
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each other. Twisted pairs are naturally susceptible to electromagnetic interference 
(EMI), the effects of which are mitigated through twisting the wires. 
A coaxial cable consists of an inner conductor and an outer conductor, separated by 
a dielectric insulating material. The inner conductor is made of a copper wire encased 
inside the dielectric material. As for the outer conductor, it is made of copper, tinned 
copper, or copper-coated steel. Typically, a coaxial cable has a characteristic imped- 
ance of 50 or 75 ohms. Compared to a twisted-pair cable, a coaxial cable offers a 
greater degree of immunity to EMI. Moreover, because of their much higher band- 
width, coaxial cables can support the transmission of digital data at much higher bit 
rates than twisted pairs. Rates up to 20 Mb/s are feasible using coaxial cables, with 
10 Mb/s being the standard. ` 

Whereas the use of a twisted pair has been confined mainly to point-to-point 
service, a coaxial cable can operate as a multiple-access medium by using high- 
impedance taps. A common application of coaxial cables is as the transmission me- 


` dium for local area networks in an office environment. 


Another common application of coaxial cables is in cable-television systems, 

also known as community-antenna television (CATV) systems. In this application 
coaxial cables are used to distribute television, audio, and data signals from the head 
end to the subscribers. The bead end is the central originating unit of the CATV 
system, where all signals are carried and processed. 
An optical fiber is a dielectric wave guide that transports light signals from one place 
to another just as a twisted-wire pair or a coaxial cable transports electrical signals. 
It consists of a central core within which the propagating electromagnetic field is 
confined and which is surrounded by a cladding layer, which is itself surrounded by 
a thin protective jacket.’ The core and cladding are both made of pure silica glass, 
whereas the jacket is made of plastic. Optical fibers have unique characteristics that 
make them highly attractive as a transmission medium. In particular, they offer the 
following unique characteristics: 


» Enormous potential bandwidth, resulting from the use of optical carrier frequen- 
cies around 2 X 10'* Hz; with such a high carrier frequency and a bandwidth 
roughly equal to 10 percent of the carrier frequency, the theoretical bandwidth of 
a lightwave system is around 2 X 101 Hz, which is very large indeed. 

> Low transmission losses, as low as 0.1-dB/km. 

> Immunity to electromagnetic interference, which is an inherent characteristic of 
an optical fiber viewed as a dielectric waveguide. 

œ Small size and weight, characterized by a diameter no greater than that of a human 

hair. 

Ruggedness and flexibility, exemplified by very high tensile strengths and the pos- 

sibility of being bent or twisted without damage. 


¥ 


Last, but by no means least, optical fibers offer the potential for low-cost line com- 
munications since they are fabricated from sand, which, unlike the copper used in 
metallic conductors, is not a scarce resource. The unique properties of optical fibers 
have fuelled phenomenal advances in lightwave systems technology, which have, in 
turn, revolutionized long-distance communications and continue to do so. 

Wireless broadcast channels support the transmission of radio and television signals. 
The information-bearing signal, representing speech, music, or pictures, is modulated 
onto a carrier frequency that identifies the transmitting station; modulation is de- 
scribed in the next section. The transmission originates from an antenna that acts as 
the transition or matching unit between the source of the modulated signal and 
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electromagnetic waves in free space. The objective in designing the antenna is to 
excite the waves in the required direction or directions, as efficiently as possible. 
Typically, the transmitting antenna is mounted on a tower to provide an unob- 
structed view of the surrounding area, as far afield as possible. By virtue of the 
phenomenon of diffraction, which is a fundamental property of wave motion, radio 
waves are bent around the earth’s surface. Propagation beyond the line of sight is 
thereby made possible, albeit with somewhat greater loss than is incurred in free 
space. 

At the receiving end, an antenna is used to pick up-the radiated waves, estab- 

lishing a communication link to the transmitter. Most radio receivers are of the 
superheterodyne type. This technique consists of down-converting the received signal 
to some convenient frequency band, called the intermediate frequency (IF) band, and 
then recovering the original information-bearing signal by means of an appropriate 
detector. 
A mobile radio channel extends the capability of the public telecommunications net- 
work by introducing mobility into the network by virtue of its ability to broadcast. 
The term mobile radio is usually meant to encompass terrestrial situations where a 
radio transmitter or receiver is capable of being moved, regardless of whether it 
actually moves or not. The major propagation effects encountered in the use of a 
mobile radio in built-up areas are due to the fact that the antenna of the mobile unit 
may lie well below the surrounding buildings. Simply put, there is no “line-of-sight” 
path for communication; rather, radio propagation takes place mainly by way of 
scattering from the surfaces of the surrounding buildings and by diffraction over and/ 
or around them. The end result is that energy reaches the receiving antenna via more 
than one path. In a mobile radio environment, we thus speak of a multipath phe- 
nomenon in that the various incoming radio waves reach their destination from 
different directions and with different time delays. Indeed, there may be a multitude 
of propagation paths with different electrical lengths, and their contributions to the 
received signal could combine in a variety of ways. Consequently, the received signal 
strength varies with location in a very complicated fashion, and so a mobile radio 
channel may be viewed as a linear time-varying channel that is statistical in nature. 
Finally, a satellite channel adds another invaluable dimension to the public telecom- 
munications network by providing broad-area coverage in both a continental and 
an intercontinental sense. Moreover, access to remote areas not covered by conven- 
tional cable or fiber communications is also a distinct feature of satellites. In almost 
all satellite communication systems, the satellites are placed in geostationary orbit. 
For the orbit to be geostationary, it has to satisfy two requirements. First, the orbit 
is geosynchronous, which requires the satellite to be at an altitude of 22,300 miles; 
a geosynchronous satellite orbits the Earth in 24 hours (i.e., the satellite is synchro- 
nous with the Earth’s rotation). Second, the satellite is placed in orbit directly above 
the equator on an eastward heading (i.c., it has zero inclination). Viewed from Earth, 
a satellite in geostationary orbit appears to be stationary in the sky. Consequently, 
an Earth station does zot have to track the satellite; rather, it merely has to point its 
antenna along a fixed direction, pointing toward the satellite. By so doing, the system 
design is simplified considerably. Communications satellites in geostationary orbit 
offer the following unique system capabilities: 


> Broad-area coverage. 
> Reliable transmission links. 
> Wide transmission bandwidths. 
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In terms of services, satellites can provide fixed point-to-point links extending over 
long distances and into remote areas, communication to mobile platforms (e.g., air- 
craft, ships), or broadcast capabilities. Indeed, communications satellites play a key 
role in the notion of the whole world being viewed as a “global village.” In a typical 
satellite communication system, a message signal is transmitted from an Earth station 
via an uplink to a satellite, amplified in a transponder (i.e., electronic circuitry) on 
board the satellite, and then retransmitted from the satellite via a downlink to another 

Earth station. With the satellite positioned in geostationary orbit, it is always visible 

to all the Earth stations located inside the satellite antenna’s coverage zones on the 

Earth’s surface. In effect, the satellite acts as a powerful repeater in the sky. The most 

popular frequency band for satellite communications is 6 GHz for the uplink and 

4 GHz for the downlink. The use of this frequency band offers the following 

attributes: 

> Relatively inexpensive microwave equipment. 

> Low attenuation due to rainfall; rainfall is a primary atmospheric cause of signal 
loss. 

> Insignificant sky background noise; the sky background noise (due to random 
noise emissions from galactic, solar, and terrestrial sources) reaches its lowest level 
between 1 and 10 GHz. 

In the 6/4-GHz band, a typical satellite is assigned a 500 MHz bandwidth that is 

divided among 12 transponders on board the satellite. Each transponder, using ap- 

proximately 36 MHz of the satellite bandwidth, corresponds to a specific radio chan- 
nel. A single transponder can carry at least one color television signal, 1200 voice 
circuits, or digital data at a rate of 50 Mb/s. 

To summarize, a communication channel is central to the operation of a com- 
munication system. Its properties determine both the information-carrying capacity 
of the system and the quality of service offered by the system. We may classify com- 
munication channels in different ways: 
> A channel may be linear or nonlinear; a wireless radio channel is linear, whereas 

a satellite channel is usually (but not always) nonlinear. 

> A channel may be time invariant or time varying; an optical fiber is time invariant, 
whereas a mobile radio channel is typically time varying. 

» A channel may be bandwidth limited or power limited (i.e., limited in the available 
transmitted power); a telephone channel is bandwidth limited, whereas an optical 
fiber link and a satellite channel are both power limited. 

Now that we have some understanding of sources of information and com- 
munication channels, we may return to the block diagram of a communication sys- 
tem shown in Figure 1. 


| Modulation Process 





The purpose of a communication system is to deliver a message signal from an information 
source in recognizable form to a user destination, with the source and the user being 
physically separated from each other. To do this, the transmitter modifies the message 
signal into a form suitable for transmission over the channel. This modification is achieved 
by means of a process known as modulation, which involves varying some parameter of 
a carrier. wave in accordance with the message signal. The receiver re-creates the original 
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message signal from a degraded version of the transmitted signal after propagation through 
the channel. This re-creation is accomplished by using a process known as demodulation, 
which is the reverse of the modulation process used in the transmitter. However, owing 
to the unavoidable presence of noise and distortion in the received signal, we find that the 
receiver cannot re-create the original message signal exactly. The resulting degradation in 
overall system performance is influenced by the type of modulation scheme used. Specifi- 
cally, we find that some modulation schemes are less sensitive to the effects of noise and 
distortion than others. 

We may classify the modulation process into continuous-wave modulation and pulse 
modulation. In continuous-wave (CW) modulation, a sinusoidal wave is used as the car- 
tier. When the amplitude of the carrier is varied in accordance with the message signal, 
we have amplitude modulation (AM), and when the angle of the carrier is varied, we have 
angle modulation. The latter form of CW modulation may be further subdivided into 
frequency modulation (FM) and phase modulation (PM), in which the instantaneous fre- 
quency and phase of the carrier, respectively, are varied in accordance with the message 
signal. 

In pulse modulation, on the other hand, the carrier consists of a periodic sequence 
of rectangular pulses. Pulse modulation can itself be of an analog or digital type. In analog 
pulse modulation, the amplitude, duration, or position of a pulse is varied in accordance 
with sample values of the message signal. In such a case, we speak of pulse-amplitude 
modulation (PAM), pulse-duration modulation (PDM), and pulse-position modulation 
(PPM). - 

The standard digital form of pulse modulation is known as pulse-code modulation 
(PCM) that has no CW counterpart. PCM starts out essentially as PAM, but with an 
important modification: The amplitude of each modulated pulse (i.e., sample of the original 
message signal) is quantized or rounded off to the nearest value in a prescribed set of 
discrete amplitude levels and then coded into a corresponding sequence of binary symbols. 
The binary symbols 0 and 1 are themselves represented by pulse signals that are suitably 
shaped for transmission over the channel. In any event, as a result of the quantization 
process, some information is always lost and the original message signal cannot therefore 
be reconstructed exactly. However, provided that the number of quantizing (discrete am- 
plitude) levels is large enough, the distortion produced by the quantization process is not 
discernible to the human ear in the case of a speech signal or the human eye in the case of 
a two-dimensional image. Among all the different modulation schemes, pulse-code mod- 
ulation has emerged as the preferred method of modulation for the transmission of analog 
message signals for the following reasons: 


> Robustness in noisy environments by regenerating the transmitted signal at regular 
intervals. 

» Flexible operation. 

® Integration of diverse sources of information into a common format. 

> Security of information in its transmission from source to destination. 


In introducing the idea of modulation, we stressed its importance as a process that 
ensures the transmission of a message signal over a prescribed channel. There is another 
important benefit, namely, multiplexing, that results from the use of modulation. Multi- 
plexing is the process of combining several message signals for their simultaneous trans- 
mission over the same channel. Three commonly used methods of multiplexing are as 
follows: 


> Frequency-division multiplexing (FDM), in which CW modulation is used to trans- 
late each message signal to reside in a specific frequency slot inside the passband of 


Analog and Digital Types of Communication 21 


the channel by assigning it a distinct carrier frequency; at the receiver, a bank of 
filters is used to separate the different modulated signals and prepare them individ- 
ually for demodulation. 

> Time-division multiplexing (TDM), in which pulse modulation is used to position 
samples of the different message signals in nonoverlapping time slots. 

> Code-division multiplexing (CDM), in which each message or is identified by a 
distinctive code. 


In FDM the message signals overlap with each other at the channel input; hence the system 
may suffer from crosstalk (i.e., interaction between message signals) if the channel is non- 
linear. In TDM the message signals use the full passband of the channel, but on a time- 
shared basis. In CDM the message signals are permitted to overlap in both time and 
frequency across the channel. 

Mention should also be made of wavelength-division multiplexing (WDM), which 
is special to optical fibers. In WDM, wavelength is used as a new degree of freedom by 
concurrently operating distinct portions of the wavelength spectrum (i.e., distinct colors) 
that are accessible within the optical fiber. However, recognizing the reciprocal relation- 
ship that exists between the wavelength and frequency of an electromagnetic wave, we 
may say that WDM is a form of FDM. 


I Analog and Digital Types of Communication 


Typically, in the design of a communication system the information source, communica- 
tion channel, and information sink (end user) are all specified. The challenge is to design 
the transmitter and the receiver with the following guidelines in mind: 


> Encode/modulate the message signal generated by the source of information, transmit 
it over the channel, and produce an “estimate” of it at the receiver output that satisfies 
the requirements of the end user. 


e Do all of this at an affordable cost. 


‘We have the option of using a digital or analog communication system. 

Consider first the case of a digital communication system represented by the block 
diagram of Figure 9, the rationale for which is rooted in information theory. The functional 
blocks of the transmitter and the receiver, starting from the far end of the channel, are 
paired as follows: 


» Source encoder-decoder. 
> Channel encoder-decoder. 
> Modulator-demodulator. 


The source encoder removes redundant information from the message signal and is re- 
sponsible for the efficient use of the channel. The resulting sequence of symbols is called 
the source code word. The data stream is processed next by the channel encoder, which 
produces a new sequence of symbols called the channel code word. The channel code word 
is longer than the source code word by virtue of the controlled redundancy built into its 
construction. Finally, the modulator represents each symbol of the channel code word by 
a corresponding analog symbol, appropriately selected from a finite set of possible analog 
symbols. The sequence of analog symbols produced by the modulator is called a waveform, 
which is suitable for transmission over the channel. At the receiver, the channel output 
(received signal) is processed in reverse order to that in the transmitter, thereby recon- 


22 


= BACKGROUND AND PREVIEW 


Estimate of 
message signa 












Message signal 









































































Source of User of 
information information 
fe ee ei ee ee ee me T 
| l 
| I 
Source l | Source 1 
encoder | decoder 
I i K | 
Source l Estimate of 
code word | | source 
l l code word 
l 1 
Transmitter Channel l l Channel | Receiver 
encoder i | decoder 
i l Estimate of ! 
1 f stimate o 
l rana [i | channel 
i code word | I iode word 
I 
| | 
Modulator | Demodulator 
l 1 
I | | 
see eee ee he ee 






Received 
signal 





Waveform 





Channel 





FIGURE 9 Block diagram of digital communication system. 


structing a recognizable version of the original message signal. The reconstructed message 
signal is finally delivered to the user of information at the destination. From this description. 
it is apparent that the design of a digital communication system is rather complex in 
conceptual terms but easy to build. Moreover, the system is robust, offering greater tol- 
erance of physical effects (e.g., temperature variations, aging, mechanical vibrations) than 
its analog counterpart, 

In contrast, the design of an analog communication system is simple in conceptual 
terms but difficult to build because of stringent requirements on linearity and system ad- 
justment. For example, voice communication requires nonlinear distortion products at 
least 40 dB below the wanted message signal. In signal-processing terms, the transmitter 
consists of a modulator and the receiver consists of a demodulator, the details of which 
are determined by the type of CW modulation used. 

The conceptual simplicity of analog communications is due to the fact that analog 
modulation techniques, exemplified by their wide use in radio and television, make rela- 
tively superficial changes to the message signal in order to prepare it for transmission over 
the channel. More specifically, there is no significant effort made by the system designer 
to tailor the waveform of the transmitted signal to suit the channel at any deeper level. 
On the other hand, digital communication theory endeavors to find a finite set of wave- 
forms that are closely matched to the characteristics of the channel and which are therefore 
more tolerant of channel impairments. In so doing, reliable communication is established 
over the channel. In the selection of good waveforms for digital communication over a 
noisy channel, the design is influenced solely by the channel characteristics. However, once 
the appropriate set of waveforms for transmission over the channel has been selected, the 
source information can be encoded into the channel waveforms, and the efficient trans- 
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mission of information from the source to the user is thereby ensured. In summary, the 
use of digital communications provides the capability for information transmission that is 
both efficient and reliable. 

From this discussion, it is apparent that the use of digital communications requires 
a considerable amount of electronic circuitry, but nowadays electronics are inexpensive, 
due to the ever-increasing availability of very-large-scale integrated (VLSI) circuits in the 
form of silicon chips. Thus although cost considerations used to be a factor in selecting 
analog communications over digital communications in the past, that is no longer the case. 

Despite the trend toward the ever-increasing use of digital communications, a strong 
case can be made for the study of analog communications for two important reasons: 


1. As long as we hear and see analog communications around us via radio and televi- 
sion, we need to understand how these communications systems work. Moreover, 
the study of analog modulation motivates other digital modulation schemes. 


2. Analog devices and circuits have a natural affinity for operating at very high speeds 
and they consume very little power compared to their digital counterparts. Accord- 
ingly, the implementation of very high-speed or very low-power communication sys- 
tems dictates the use of an analog approach. 


| Shannon's Information Capacity Theorem 


The goal of a communication system designer is to configure a system that transports a 
message signal from a source of interest across a noisy channel to a user at the other end 
of the channel with the following objective: 


The message signal is delivered to the user both efficiently and reliably, subject to 
certain design constraints: allowable transmit power, available channel bandwidth, 
and affordable cost of building the system. 


In the case of a digital communication system, reliability is commonly expressed in terms 
of bit error rate (BER) or probability of bit error measured at the receiver output. Clearly, 
the smaller the BER, the more reliable the communication system is. A question that comes 
to mind in this context is whether it is possible to design a communication system that 
operates with zero BER even through the channel is noisy. In an ideal setting, the answer 
to this question is an emphatic yes. The answer is embodied in one of Shannon’s celebrated 
theorems,’° which is called the information capacity theorem. 

Let B denote the channel bandwidth, and let SNR denote the received signal-to-noise 
ratio. The information capacity theorem states that ideally these two parameters are related 
as 


C = B log,(1 + SNR) b/s (1) 


where C is the information capacity of the channel. The information capacity is defined 
as the maximum rate at which information can be transmitted across the channel without 
error; it is measured in bits per second (b/s). For a prescribed channel bandwidth B and 
received SNR, the information capacity theorem tells us that a message signal can be 
transmitted through the system without error even when the channel is noisy, provided 
that the actual signaling rate R in bits per second, at which data are transmitted through 
the channel, is less than the information capacity C. 
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Unfortunately, Shannon’s information capacity theorem does not tell us how to de- 
sign the system. Nevertheless, from a design point of view, the theorem is very valuable 
for the following reasons: 


1. The information capacity theorem provides a bound on what rate of data transmis- 
sion is theoretically attainable for prescribed values of channel bandwidth B and 
received SNR. On this basis, we may use the ratio 


1-6 


as a measure of the efficiency of the digital communication system under study. The 
closer 7 is to unity, the more efficient the system is. 

2. Equation (1) provides a basis for the trade-off between channel bandwidth B and 
received SNR. In particular, for a prescribed signaling rate R, we may reduce the 
required SNR by increasing the channel bandwidth B, hence the motivation for using 
a wideband modulated scheme (e.g., pulse-code modulation) for improved noise 
performance. 

3. Equation (1) provides an idealized framework for comparing the noise performance 
of one modulation scheme against another. 


| A Digital Communication Problem 





When we speak of a digital communication system having a low bit error rate, say, the 
implication is that only a small fraction in a long stream of binary symbols is decoded in 
error by the receiver. The issue of the receiver determining whether a binary symbol sent 
over a noisy channel is decoded in error or not is of fundamental importance to the design 
of digital communication systems. It is therefore appropriate briefly to discuss this basic 
issue so as to motivate the study of communication systems. 

Suppose we have a random binary signal, m(t), consisting of symbols 1 and 0. that 
are equally likely. Symbol 1 is represented by a constant level +1, and symbol 0 is rep- 
resented by a constant level —1, each of which lasts for a duration T. Such a signal may 
represent the output of a digital computer or the digitized version of a speech signal. To 
facilitate the transmission of this signal over a communication channel, we employ a simple 
modulation scheme known as phase-shift keying. Specifically, the information bearing 
signal #(t) is multiplied by a sinusoidal carrier wave A, cos(27f,t), where A, is the carrier 
amplitude, f, is the carrier frequency, and ¢ is time. Figure 10a shows.a block diagram of 
the transmitter, the output of which is defined by 


is cos(27f.t) for symbol 1 


S —A, cos(27f.t) for symbol 0 


(2) 
where 0 = tT. The carrier frequency f, is a multiple of 1/T. 

The channel is assumed to be distortionless but noisy, as depicted in Figure 10b. The 
received signal x(t) is thus defined by 


x(t) = s(t) + w(t) (3) 


where w(t} is the additive channel noise. 
The receiver consists of a correlator followed by a decision-making device, as de- 
picted in Figure 10c. The correlator multiplies the received signal x(t) by a locally generated 
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Message Transmitted Transmitted z Channel output 
signal m(s) signal s(¢) signal s{9 x (received signal) x4) 
a + 
Carrier wave Noise 
A, cos (27rf.2) w(t) 
{a) (b) 
Correlator 
Received pct Say 1ifyp>0 
signal x() manng 


Otherwise, say 0 





Local carrier 
cos (2rf,t) Threshold = 0 


(c) 


FIGURE 10 Elements of a digital communication system. (a) Block diagram of transmitter. 
(b) Block diagram of channel. (c) Block diagram of receiver. 


carrier cos(27f.t) and then integrates the product over the symbol interval 0 = t = T, 
producing the output 


T 
Yr = Í x(t) cos(27f,t) dt (4) 


Substituting Equations (2) and (3) into (4) and invoking the assumption that the carrier 
frequency f, is a multiple of 1/T, we obtain (after the simplification of terms) 


A +wr for symbol 1 
Yr = (5) 


-4 +wr for symbol 0 


where wp is the contribution of the correlator output due to the channel noise w(t). To 
reconstruct the original binary signal m(t), the correlator output yz is compared against a 
threshold of zero volts by the decision-making device, the operation of which is based on 
the following rule: 


If the correlator output yr is greater than zero, the receiver outputs symbol 1; 
otherwise, it outputs symbol 0. 


With this background, we may now discuss/raise some basic issues. First, from Fou- 
tier analysis we find that the time-bandwidth product of a pulse signal is constant. This 
means that the bandwidth of a rectangular pulse of duration T is inversely proportional 
to T. The transmitted signal in Figure 10a consists of the product of this rectangular signal 
and the sinusoidal carrier A, cos(27f.t). The multiplication of a signal by a sinusoid has 
the effect of shifting the Fourier transform of the signal to the right by f. and to the left 
by an equal amount, except for the scaling factor of 1/2. It follows therefore that the 
bandwidth of the transmitted signal m(t), and therefore the required channel bandwidth, 
is inversely proportional to the reciprocal of the symbol duration T. For the problem at 
hand, the reciprocal of T is also the signaling rate of the system in b/s. 
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There are, however, some other issues that require theoretical considerations: 


1. What is the justification for the receiver structure of Figure 10c? 

2. The noise contribution w7 is the value of a random variable W produced by sampling 
a certain realization w(t) of the channel noise at time t = T in accordance with 
Equations (3) and (4). How do we relate the statistics of the random variable W to 
the statistical characteristics of the channel noise? 

3. The receiver of Figure 10c makes occasional errors due to the random nature of the 
correlator output. That is, the receiver decides in favor of symbol 0 given that symbol 
1 was actually transmitted, and vice versa. What is the probability of decision errors? 


Moreover, there are some important practical issues that need attention: 


1. Channel bandwidth is a highly valuable resource. How do we choose a modulation 
scheme that conserves bandwidth in a cost-effective manner? : 
The binary signal m(t) may include redundant symbols introduced into it throu 
the use of channel encoding so as to provide protection against channel noise. How 
do we design the channel encoder in the transmitter and the channel decoder in the 
receiver so as to come very close to Shannon’s information capacity theorem in a 
physically realizable manner? 

The locally generated carrier in the receiver of Figure 10c is physically separate from 
the carrier source used for modulation in the transmitter. How do we synchronize 
the receiver to the transmitter with respect to both the carrier phase and symbol 
timing so as to justify the use of Equation (4) as the basis of decision-making in the 
reconstruction of the original binary signal? 


2 


w 


The theoretical and practical issues raised here in the context of the simple digital 
communication system of Figure 10 are addressed in the following chapters of the book. 


Í Historical Notes!! 


A preview of communications would be incomplete without a history of the subject. In 
this final section of this introductory chapter we present some historical notes on com- 
munications; each paragraph focuses on some important and related events. It is hoped 
that this material will provide a sense of inspiration and motivation for the reader. 

In 1837, the telegraph was perfected by Samuel Morse, a painter. With the words 
“What hath God wrought,” transmitted by Morse’s electric telegraph between Washing- 
ton, D.C., and Baltimore, Maryland, in 1844, a completely revolutionary means of real- 
time, long-distance communications was triggered. The telegraph is the forerunner of dig- 
ital communications in that the Morse code is a variable-length ternary code using an 
alphabet of four symbols: a dot, a dash, a letter space, and a word space; short sequences 
represent frequent letters, whereas long sequences represent infrequent letters. This type 
of signaling is ideal for manual keying. Subsequently, Emile Baudot developed a fixed- 
length binary code for telegraphy in 1875. In Baudot’s telegraphic code, well-suited for 
use with teletypewriters, each code word consists of five equal-length code elements, and 
each element is assigned one of two possible states: a mark or a space {i.e., symbol 1 or 0 
in today’s terminology). 

In 1864, James Clerk Maxwell formulated the electromagnetic theory of light and 
predicted the existence of radio waves; the underlying set of equations bears his name. The 
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existence of radio waves was established experimentally by Heinrich Hertz in 1887. In 
1894, Oliver Lodge demonstrated wireless communication over a relatively short distance 
(150 yards). Then, on December 12, 1901, Guglielmo Marconi received a radio signal at 
Signal Hill in Newfoundland; the radio signal had originated in Cornwall, England, 1700 
miles away across the Atlantic. The way was thereby opened toward a tremendous broad- 
eming of the scope of communications. In 1906, Reginald Fessenden, a self-educated aca- 
demic, made history by conducting the first radio broadcast. 

In 1875, the telephone was invented by Alexander Graham Bell, a teacher of the 
deaf. The telephone made real-time transmission of speech by electrical encoding and 
replication of sound a practical reality. The first version of the telephone was crude and 
weak, enabling people to talk over short distances only. When telephone service was only 
a few years old, interest developed in automating it. Notably, in 1897, A. B. Strowger, an 
undertaker from Kansas City, Missouri, devised the automatic step-by-step switch that 
bears his name; of all the electromechanical switches devised over the years, the Strowger 
switch was the most popular and widely used. 

In 1904, John Ambrose Fleming invented the vacuum-tube diode, which paved the 
way for the invention of the vacuum-tube triode by Lee de Forest in 1906. The discovery 
of the triode was instrumental in the development of transcontinental telephony in 1913 
and signaled the dawn of wireless voice communications. Indeed, until the invention and 
perfection of the transistor, the triode was the supreme device for the design of electronic 
amplifiers. 

In 1918, Edwin H. Armstrong invented the superbeterodyne radio receiver; to this 
day, almost all radio receivers are of this type. In 1933, Armstrong demonstrated another 
revolutionary concept, namely, a modulation scheme that he called frequency modulation 
(FM); Armstrong’s paper making the case for FM radio was published in 1936. 

The first all-electronic television system was demonstrated by Philo T. Farnsworth 
in 1928, and then by Vladimir K. Zworykin in 1929. By 1939, the British Broadcasting 
Corporation (BBC) was broadcasting television on a commercial basis. 

In 1928, Harry Nyquist published a classic paper on the theory of signal transmission 
in telegraphy. In particular, Nyquist developed criteria for the correct reception of tele- 
graph signals transmitted over dispersive channels in the absence of noise. Much of Ny- 
quist’s early work was applied later to the transmission of digital data over dispersive 
channels. 

In 1937, Alec Reeves invented pulse-code modulation (PCM) for the digital encoding 
of speech signals. The technique was developed during World War II to enable the en- 
cryption of speech signals; indeed, a full-scale, 24-channel system was used in the field by 
the United States military at the end of the war. However, PCM had to await the discovery 
of the transistor and the subsequent development of large-scale integration of circuits for 
its commercial exploitation. 

In 1943, D. O. North devised the matched filter for the optimum detection of a 
known signal in additive white noise. A similar result was obtained in 1946 independently 
by J. H. Van Vleck and D. Middleton, who coined the term matched filter. 

In 1947, the geometric representation of signals was developed by V. A. Kotel’nikov 
in a doctoral dissertation presented before the Academic Council of the Molotov Energy 
Institute in Moscow. This method was subsequently brought to full fruition by John M. 
Wozencraft and Irwin M. Jacobs in a landmark textbook published in 1965. 

In 1948, the theoretical foundations of digital communications were laid by Claude 
Shannon in a paper entitled “A Mathematical Theory of Communication.” Shannon’s 
paper was received with immediate and enthusiastic acclaim. It was perhaps this response 
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that emboldened Shannon to amend the title of his paper to “The Mathematical Theory 
of Communication” when it was reprinted a year later in a book co-authored with Warren 
Weaver. It is noteworthy that prior to the publication of Shannon’s 1948 classic paper, it 
was believed that increasing the rate of information transmission over a channel would 
increase the probability of error; the communication theory community was taken by 
surprise when Shannon proved that this was not true, provided that the transmission rate 
was below the channel capacity. Shannon’s 1948 paper was followed by some significant 
advances in coding theory, which include the following: 


> Development of the first nontrivial error-correcting codes by M. J. E. Golay in 1949 
and Richard W. Hamming in 1950. 

® Development of turbo codes by C. Berrou, A. Glavieux, and P. Thitimajshima in 
1993; turbo codes provide near-optimum error-correcting coding and decoding per- 
formance in the Shannon sense. 


The transistor was invented in 1948 by Walter H. Brattain, John Bardeen, and Wil- 
liam Shockley at Bell Laboratories. The first silicon integrated circuit (IC) was produced 
by Robert Noyce in 1958. These landmark innovations in solid-state devices and integrated 
circuits led to the development of very-large-scale integrated (VLSI) circuits and single- 
chip microprocessors, and with them the nature of signal processing and the telecommu- 
nications industry changed forever. 

The invention of the transistor in 1948 spurred the application of electronics to 
switching and digital communications. The motivation was to improve reliability, increase 
capacity, and reduce cost. The first call through a stored-program system was placed in 
March 1958 at Bell Laboratories, and the first commercial telephone service with digital 
switching began in Morris, Illinois, in June 1960. The first T-1 carrier system transmission 
was installed in 1962 by Bell Laboratories. 

During the period 1943 to 1946, the first electronic digital computer, called the 
ENIAC, was built at the Moore School of Electrical Engineering of the University of Penn- 
sylvania under the technical direction of J. Presper Eckert, Jr., and John W. Mauchly. 
However, John von Neumann’s contributions were among the earliest and most funda- 
mental to the theory, design, and application of digital computers, which go back to the 
first draft of a report written in 1945. Computers and terminals started communicating 
with each other over long distances in the early 1950s. The links used were initially voice- 
grade telephone channels operating at low speeds (300 to 1200 b/s). Various factors have 
contributed to a dramatic increase in data transmission rates; notable among them are the 
idea of adaptive equalization, pioneered by Robert Lucky in 1965, and efficient modula- 
tion techniques, pioneered by G. Ungerboeck in 1982. Another idea widely employed in 
computer communications is that of automatic repeat-request (ARQ). The ARQ method 
was originally devised by H. C. A. van Duuren during World War II and published in 
1946. It was used to improve radio-telephony for telex transmission over long distances. 

From 1950 to 1970, various studies were made on computer networks. However, 
the most significant of them in terms of impact on computer communications was the 
Advanced Research Project Agency Network (ARPANET), first put into service in 1971. 
The development of ARPANET was sponsored by the Advanced Research Projects Agency 
of the U.S. Department of Defense. The pioneering work in packet switching was done on 
ARPANET. In 1985, ARPANET was renamed the Internet. The turning point in the evo- 
lution of the Internet occurred in 1990 when Tim Berners-Lee proposed a hypermedia 
software interface to the Internet, which he named the World Wide Web.” Thereupon, in 
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the space of only about two years, the Web went from nonexistence to worldwide popu- 
larity, culminating in its commercialization in 1994. How do we explain the explosive 
growth of the Internet? We may answer this question by offering these reasons:!3 


> Before the Web exploded into existence, the ingredients for its creation were already 
in place. In particular, thanks to VLSI, personal computers (PCs) had already become 
ubiquitous in homes throughout the world, and they were increasingly equipped with 
modems for interconnectivity to the outside world. 

> For about two decades, the Internet had grown steadily (albeit within a confined 
community of users), reaching a critical threshold of user-value based electronic mail 
and file transfer. 

> Standards for document description and transfer, hypertext markup language 
(HTML), and hypertext transfer protocol (HTTP) had been adopted. 


Thus, everything needed for creating the Web was already in place except for two critical 
ingredients: a simple user interface and a brilliant service concept. 

In 1955, John R. Pierce proposed the use of satellites for communications. This 
proposal was preceded, however, by an earlier paper by Arthur C. Clark that was pub- 
lished in 1945, also proposing the idea of using an Earth-orbiting satellite as a relay point 
for communication between two Earth stations. In 1957, the Soviet Union launched Sput- 
nik I, which transmitted telemetry signals for 21 days. This was followed shortly by the 
launching of Explorer I by the United States in 1958, which transmitted telemetry signals 
for about five months. A major experimental step in communications satellite technology 
was taken with the launching of Telstar I from Cape Canaveral on July 10, 1962. The 
Telstar satellite was built by Bell Laboratories, which had acquired considerable knowl- 
edge from pioneering work by Pierce. The satellite was capable of relaying TV programs 
across the Atlantic; this was made possible only through the user of maser receivers and 
large antennas. 

The use of optical means (e.g., smoke and fire signals) for the transmission of infor- 
mation dates back to prehistoric times. However, no major breakthrough in optical com- 
munications was made until 1966, when K. C. Kao and G. A. Hockham of Standard 
Telephone Laboratories, U.K., proposed the use of a clad glass fiber as a dielectric wave- 
guide. The /aser (an acronym for light amplification by stimulated emission of radiation) 
had been invented and developed in 1959 and 1960. Kao and Hockham pointed out that 
(1) the attenuation in an optical fiber was due to impurities in the glass, and (2) the intrinsic 
loss, determined by Rayleigh scattering, is very low. Indeed, they predicted that a loss of 
20 dB/km should be attainable. This remarkable prediction, made at a time when the 
power loss in a glass fiber was about 1000 dB/km, was to be demonstrated later. Nowa- 
days, transmission losses as low as 0.1 dB/km are achievable. 

The spectacular advances in microelectronics, digital computers, and lightwave sys- 
tems that we have witnessed to date, and that will continue into the future, are all re- 
sponsible for dramatic changes in the telecommunications environment; many of these 
changes are already in place, and more changes will evolve as time goes on. 


i NOTES AND REFERENCES 





1. For essays on an early account of communications and other related disciplines (e.g., elec- 
tronics, computers, radar, radio astronomy, satellites), see Overhage (1962); in particular, 
see the chapter on “Communications” by L. V. Berkner, pp. 35-50. 


30 


© BACKGROUND AND PREVIEW 


2: 


The JPEG image coding standard is discussed in the papers by Wallace (1991); see also the 
article by T. A. Ramstad in the handbook edited by Madisetti and Williams (1998). 


. The discrete cosine transform (DCT) and its inverse for a block of 8 x 8 source image 


samples are respectively defined by 


Flu, v) = Z CCW) [>> > fixy) cos( 22 tie) cos 22 #10) | 


x=0 y=0 
7 OF 
f(x, y) = [2 > Clu) Civ) Flu, v) cos( 22 tt i iwr) cos 224 = ta) 
4=0 v=0 16 16 
where 


1 x 
Cu), Cv) = 175 for u = 0 andv = 0 


1 otherwise 


For a full treatment of the DCT, see Rao and Yip (1990). 


. The MPEG-1 video coding standard is discussed in the paper by Gall (1991); see also the 


article by A. M. Tekalp in the handbook edited by Madisetti and Williams (1998), which 
discusses the follow-up versions of the MPEG video coding standard. 


. The MPEG-1 audio coding standard is discussed in the papers by Brandenburg and Stoll 


(1994) and Pan (1993); see also the article by P. Noll in the handbook edited by Madisetti 
and Williams (1998), which also discusses the follow-up versions of the MPEG audio 
coding standard. In particular, the widespread use of the more current standard, MPEG-3 
audio, is resulting in a level of piracy that may dwarf the earlier problems of “bootleg” 
cassette tapes. 


6. For a detailed discussion of communication networks, see Tanenbaum (1996). 


7. The OSI reference model was developed by a subcommittee of the International Organi- 


10. 


11. 
12. 
13. 


zation for Standardization (ISO) in 1977. For a discussion of the principles involved in 
arriving at the seven layers of the OSI model and a description of the layers themselves, 
see Tanenbaum (1996). 


. SONET was originally proposed by Telcordia Technologies Inc. (then known as Bellcore) 


and standardized by the American National Standards Institute (ANSI). Later, CCITT 
approved a SONET standard and issued a set of parallel recommendations called synchro- 
nous digital hierarchy (SDH). The differences between SONET and SDH are of a minor 
nature. 


. For a thorough and precise analysis of the propagation of light waves in an optical fiber, 


we need to treat it as a dielectric waveguide and use Maxwell’s equations to carry out the 
analysis; such an analysis is highly mathematical in nature. For a readable account of the 
analysis, see Chapter 3 of Green, Jr. (1993). 


For a semitechnical overview of Shannon’s theorems on information theory presented in a 
highly readable fashion, see the book entitled Silicon Dreams by Lucky (1989). 


For a readable account of the history of communications, see Lebow (1995). 
For a historical account of the development of the Internet, see Leiner et al. (1997). 


For an insightful essay on new telecommunications services and how society reacts to their 
development, see Lucky (1997). This paper points to Metcalf’s law, according to which it 
seems as if any new telecommunications service must take a Jong time for it to build to 
universal acceptance. Lucky cites the World Wide Web as a startling counterexample to 
Metcalf’s law and gives the reasons why. 





RANDOM PROCESSES 


This chapter presents an introductory treatment of stationary random processes with 
emphasis on second-order statistics. In particular, it discusses the following issues: 


b> The notion of a.random process. 
> The requirement that has to be satisfied for a random process to be stationary. 


> The partial description of a random process in terms of its mean, correlation, and 
covariance functions. 


» The conditions that have to be satisfied for a stationary random process to be ergodic, a 
property that enables us to substitute time averages for ensemble averages. 


» What happens to a stationary random process when it is transmitted through a linear 
time-invariant filter? 


> The frequency-domain description of a random process in terms of power spectral density. 
» The characteristics of an important type of random process known as a Gaussian process. 
> Sources of noise and their narrowband form. 


> Rayleigh and Rician distributions, which represent two special probability distributions 
that arise in the study of communication systems. 


i L.l Introduction 


The idea of a mathematical model used to describe a physical phenomenon is well estab- 
lished in the physical sciences and engineering. In this context, we may distinguish two 
classes of mathematical models: deterministic and stochastic. A model is said to be deter- 
ministic if there is no uncertainty about its time-dependent behavior at any instant of time. 
However, in many real-world problems the use of a deterministic model is inappropriate 
because the physical phenomenon of interest involves too many unknown factors, Nev- 
ertheless, it may be possible to consider a model described in probabilistic terms in that 
we speak of the probability of a future value lying between two specified limits. In such a 
case, the model is said to be stochastic or random. A brief review of probability theory is 
presented in Appendix 1. 

Consider, for example, a radio communication system. The received signal in such 
a system usually consists of an information-bearing signal component, a random interfer- 
ence component, and channel noise. The information-bearing signal component may rep- 
resent, for example, a voice signal that, typically, consists of randomly spaced bursts of 
energy of random duration. The interference component may represent spurious electro- 
magnetic waves produced by other communication systems operating in the vicinity of the 
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radio receiver. A major source of channel noise is thermal noise, which is caused by the 
random motion of the electrons in conductors and devices at the front end of the receiver. 
We thus find that the received signal is random in nature. Although it is not possible to 
predict the exact value of the signal in advance, it is possible to describe the signal in terms 
of statistical parameters such as average power and power spectral density, as discussed 
in this chapter. 


1.2 Mathematical Definition 


of a Random Process 








In light of these introductory remarks, it is apparent that random processes have two 
properties. First, they are functions of time. Second, they are random in the sense that 
before conducting an experiment, it is not possible to exactly define the waveforms that 
will be observed in the future. 

In describing a random experiment it is convenient to think in terms of a sample 
space. Specifically, each outcome of the experiment is associated with a sample point. The 
totality of sample points corresponding to the aggregate of all possible outcomes of the 
experiment is called the sample space. Each sample point of the sample space is a function 
of time. The sample space or ensemble composed of functions of time is called a random 
or stochastic process.’ As an integral part of this notion, we assume the existence of a 
probability distribution defined over an appropriate class of sets in the sample space, so 
that we may speak with confidence of the probability of various events. 

Consider, then, a random experiment specified by the outcomes s from some sample 
space S, by the events defined on the sample space S, and by the probabilities of these 
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FIGURE 1.1 An ensemble of sample functions. 
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events. Suppose that we assign to each sample point s a function of time in accordance 
with the rule: 


X(t, s), -T=r=T (1.1) 


where 2T is the total observation interval. For a fixed sample point s,, the graph of the 
function X(t, s;) versus time ¢ is called a realization or sample function of the random 
process. To simplify the notation, we denote this sample function as 


x,(t) = X(t, s) (1.2) 


Figure 1.1 illustrates a set of sample functions {x,(t)|j = 1, 2,..., n}. From this figure, 
we note that for a fixed time ¢, inside the observation interval, the set of numbers 


{x1(tz), Xa(te), sees Xnlte)} z {X(t $1), X (ty, Sa), eas | X(t, Sn)} 


constitutes a random variable. Thus we have an indexed ensemble (family) of random 
variables {X(t, s)}, which is called a random process. To simplify the notation, the custom- 
ary practice is to suppress the s and simply use X(t) to denote a random process. We may 
now formally define a random process X(t) as an ensemble of time functions together with 
a probability rule that assigns a probability to any meaningful event associated with an 
observation of one of the sample functions of the random process. Moreover, we may 
distinguish between a random variable and a random process as follows: 


> For a random variable, the outcome of a random experiment is mapped into a 
number. 

» For a random process, the outcome of a random experiment is mapped into a wave- 
form that is a function of time. 


| 1.3 Stationary Processes 


In dealing with random processes encountered in the real world, we often find that the 
statistical characterization of a process is independent of the time at which observation of 
the process is initiated. That is, if such a process is divided into a number of time intervals, 
the various sections of the process exhibit essentially the same statistical properties. Such 
a process is said to be stationary. Otherwise, it is said to be nonstationary. Generally 
speaking, a stationary process arises from a stable physical phenomenon that has evolved 
into a steady-state mode of behavior, whereas a nonstationary process arises from an 
unstable phenomenon. 

To be more precise, consider a random process X(t) that is initiated at t = —%. Let 
X(t), X(t2),..., X(t,) denote the random variables obtained by observing the random 
process X(t) at times tı, tz, . . . , t, respectively. The joint distribution function of this set 
of random variables is Fxe,),..., x(e(%1» - - - » X4). Suppose next we shift all the observation 
times by a fixed amount 7, thereby obtaining a new set of random variables X(t, + 7), 
X(t, + 7),..., X(t, + 7). The joint distribution function of this latter set of random 
variables is Fu +7),..., x(e+7)(%1s + » -> Xg). The random process X(t) is said to be stationary 
in the strict sense or strictly stationary if the following condition holds: 


Fyn... xini wey Xe) = Fyey,..., xe) (X15 aeey Xe) (1.3) 
for all time shifts 7, all k, and all possible choices of observation times t4, . . . , tg. In other 
words, a random process X(t), initiated at time t = —®, is strictly stationary if the joint 


distribution of any set of random variables obtained by observing the random process X(t) 
is invariant with respect to the location of the origin t = 0. Note that the finite-dimensional 
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distributions in Equation (1.3) depend on the relative time separation between random 
variables but not on their absolute time. That is, the random process has the same prob- 
abilistic behavior through all time. 

Similarly, we may say that two random processes X(t) and Y(t) are jointly strictly 
stationary if the joint finite-dimensional distributions of the two sets of random variables 
X(t), -- - , X(t) and Y(¢{),..., Y(t}) are invariant with respect to the origin t = 0 for all 
k and j and all choices of observation times t),..., tę and ti, «+, tje 

Returning to Equation (1.3), we may distinguish two situations of special interest: 


1. For k = 1, we have 
Fxn(x) = Fxe+alx) = Fy(x) for all ż and 7 (1.4) 
That is, the first-order distribution function of a stationary random process is inde- 
pendent of time. 
2. For k = 2 and 7 = —t,, we have 
Pye xe) (*1s x2) = Py) x(e,—1,) (15 x2) for all ż4 and ż, (1.5) 


That is, the second-order distribution function of a stationary random process de- 
pends only on the time difference between the observation times and not on the 
particular times at which the random process is observed. 


These two properties have profound implications for the statistical parameterization of a 
stationary random process; this issue is discussed in Section 1.4, 


& EXAMPLE 1.1 


Consider Figure 1.2, depicting three spatial windows located at times t4, t2, t3. We wish to 
evaluate the probability of obtaining a sample function x(t) of a random process X(t) that 
passes through this set of windows, that is, the probability of the joint event 


A = fm; < X(t) = bh i= 1,2,3 
In terms of the joint distribution function, this probability equals 
P(A) = Fy), X(e),.Xttg)(Pay bz, b3) - Foie), xan xilt 2, a3) 


Suppose now the random process X(¢) is known to be strictly stationary. An implication 
of strict stationarity is that the probability of the set of sample functions of this process passing 
through the windows of Figure 1.32 is equal to the probability of the set of sample functions 
passing through the corresponding time-shifted windows of Figure 1.3b. Note, however, that 
it is not necessary that these two sets consist of the same sample functions. <q 
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FIGURE 1.2 Illustrating the probability of a joint event. 
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FIGURE 1.3 Illustrating the concept of stationarity in Example 1.1. 


1.4 Mean, Correlation, 
and Covariance Functions 


Consider a strictly stationary random process X(t). We define the mean of the process X(t) 
as the expectation of the random variable obtained by observing the process at some time 
t, as shown by 


x(t) = E[X(¢)] 


á (1.6) 
= E xfrxin(x) dx 
where fxm (x) is the first-order probability density function of the process. From Equation 
(1.4) we deduce that for a strictly stationary random process, fx (x) is independent of 
time £. Consequently, the mean of a strictly stationary process is a constant, as shown by 


bex(t) = px for all t (1.7) 


We define the autocorrelation function of the process X(t) as the expectation of the product 
of two random variables, X(t) and X(t2), obtained by observing the process X(t) at times 
tı and t, respectively. Specifically, we write 


Rxy(ty, t2) = E[X()X(é)] 
o pæ (1.8) 
= 1 f x1%2 fxi) xe (X1 x2) dxıdxz 


where fxe xelX1s X2) is the second-order probability density function of the process. 
From Equation (1.5), we deduce that for a strictly stationary random process, 
Fxie,),xe)(%1, x2) depends only on the difference between the observation times ¢, and tz. 
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This, in turn, implies that the autocorrelation function of a strictly stationary process 
depends only on the time difference t, — t,, as shown by 


Rxlti, 6) = Rx(t — t) for all t, and t, (1.9) 
Similarly, the autocovariance function of a strictly stationary process X(t) is written as 


Cx(t to) = El(X(t1) — ux)(X(&) — ex) 
= Rxlt, — th) — Bk 
Equation (1.10) shows that, like the autocorrelation function, the autocovariance function 
of a strictly stationary process X(t) depends only on the time difference t — tı. This 
equation also shows that if we know the mean and autocorrelation function of the process, 
we can uniquely determine the autocovariance function. The mean and autocorrelation 
function are therefore sufficient to describe the first two moments of the process. 
However, two important points should be carefully noted: 


(1.10) 


1. The mean and autocorrelation function only provide a partial description of the 
distribution of a random process X(t). 

2. The conditions of Equations (1.7) and (1.9), involving the mean and autocorrelation 
function, respectively, are not sufficient to guarantee that the random process X(t) 
is strictly stationary. 


Nevertheless, practical considerations often dictate that we simply limit ourselves to a 
partial description of the process given by the mean and autocorrelation function. The 
class of random processes that satisfy Equations (1.7) and (1.9) has been given various 
names, such as second-order stationary, wide-sense stationary, or weakly stationary pro- 
cesses. Henceforth, we shall simply refer to them as stationary processes.” 

A stationary process is not necessarily strictly stationary because Equations (1.7) and 
(1.9) obviously do not imply the invariance of the joint (k-dimensional) distribution of 
Equation (1.3) with respect to the time shift 7 for all k. On the other hand, a strictly 
stationary process does not necessarily satisfy Equations (1.7) and (1.9) as the first- and 
second-order moments may not exist. Clearly, however, the class of strictly stationary 
processes with finite second-order moments forms a subclass of the class of all stationary 
processes. 


PROPERTIES OF THE AUTOCORRELATION FUNCTION 


For convenience of notation, we redefine the autocorrelation function of a stationary pro- 
cess X(t) as 


Rx(t) = E[X(t + )X(¢)] for allt (1.11) 
This autocorrelation function has several important properties: 


1. The mean-square value of the process may be obtained from R;,(7) simply by putting 
7 = 0 in Equation (1.11), as shown by 


Rx(0) = E[X*(2)] (1.12) 
2. The autocorrelation function R;(7)-is an even function of 7, that is, 
Rx(7) = Rx(~—7) (1.13) 


This property follows directly from the defining equation (1.11). Accordingly, we 
may also define the autocorrelation function Rx(7) as 


Rx(t) = E[X(t)X(é — 7] 
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FIGURE 1.4 Illustrating the autocorrelation functions of slowly and rapidly fluctuating random 
processes, 


3. The autocorrelation function Rx(7) has its maximum magnitude at 7 = 0, that is, 
|Rx(7)| = Rx(0) (1.14) 

To prove this property, consider the nonnegative quantity 

E[(X(¢ + 7) + X(t))?] = 0 
Expanding terms and taking their individual expectations, we readily find that 

E[X2(t + 7)] + 2E[X(t + 7)X(t)] + E[X(t)] = 0 
which, in light of Equations (1.11) and (1.12), reduces to 
2Rx(0) + 2Rx(7) = 0 

Equivalently, we may write 

—Rx(0) = Rx(7) = Rx(0) 
from which Equation (1.14) follows directly. 


The physical significance of the autocorrelation function Rx(7) is that it provides a 
means of describing the interdependence of two random variables obtained by observing 
a random process X(t) at times 7 seconds apart. It is therefore apparent that the more 
rapidly the random process X(t) changes with time, the more rapidly will the autocorre- 
lation function Rx(7) decrease from its maximum Rx(0) as 7 increases, as illustrated in 
Figure 1.4. This decrease may be characterized by a decorrelation time To, such that for 
T> To, the magnitude of the autocorrelation function Rx(7) remains below some prescribed 
value. We may thus define the decorrelation time 7 of a stationary process X(t) of zero 
mean as the time taken for the magnitude of the autocorrelation function Rx(7) to decrease 
to 1 percent, say, of its maximum value Rx(0). 


> EXAMPLE 1.2 Sinusoidal Wave with Random Phase 
Consider a sinusoidal signal with random phase, defined by 
X(t) = A cos(2mft + ©) (1.15) 


where A and f, are constants and © is a random variable that is uniformly distributed over 
the interval [—7, a], that is, 


1 
fol) = Fa See te (1.16) 
0, elsewhere 
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FIGURE 1.5 Autocorrelation function of a sine wave with random phase. 


This means that the random variable ® is equally likely to have any value 6 in the interval 
[- 7, a]. Each value of @ corresponds to a sample in the sample space of the random process 
X. 

The process X(t) defined by Equations (1.15) and (1.16) may represent a locally gen- 
erated carrier in the receiver of a communication system, which is used in demodulation of 
the received signal. In particular, the random variable @ denotes the phase difference between 
this locally generated carrier and the sinusoidal carrier wave used to modulate the message 
signal in the transmitter. 

The autocorrelation function of X(t) is 


Rx(7) = E[X(t + 7)X(0)] 
= E[A? cos(2mft + 2af.r + @) cos(2mf,t + @)] 


A? A? 
ET Elcos(4aft + 2af.7 + 20)] + 5 E[cos(27f,7)] 


Az 


m 2 
2 Í ee cos(4af.t + 2af.r + 28) do + = cos(27f.7) 


a 2a 
The first term integrates to zero, and so we get 
AZ 
Rx(1) = > cos(2af-7) (1.17) 
which is plotted in Figure 1.5. We see therefore that the autocorrelation function of a sinu- 


soidal wave with random phase is another sinusoid at the same frequency in the “7 domain” 
rather than the original time domain. q 


> EXAMPLE 1.3 Random Binary Wave 


Figure 1.6 shows the sample function x(t) of a process X(t) consisting of a random sequence 
of binary symbols 1 and 0. The following assumptions are made: 


1. The symbols 1 and 0 are represented by pulses of amplitude +A and —A volts, respec- 
tively, and duration T seconds. 

2. The pulses are not synchronized, so the starting time £4 of the first complete pulse for 
positive time is equally likely to lie anywhere between zero and T seconds. That is, tz 
is the sample value of a uniformly distributed random variable Tz, with its probability 
density function defined by : 


1 
fit) = 47? OS ST 
0, elsewhere 


3. During any time interval (2 — 1)T < t — ty < nT, where 7 is an integer, the presence 
of a 1 or a 0 is determined by tossing a fair coin; specifically, if the outcome is heads, 
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FIGURE 1.6 Sample function of random binary wave. 


we have a 1 and if the outcome is tails, we have a 0. These two symbols are thus equally 
likely, and the presence of a 1 or O in any one interval is independent of all other 
intervals. 


Since the amplitude levels —A and +A occur with equal probability, it follows imme- 
diately that E[X(t}] = 0 for all t, and the mean of the process is therefore zero. 

To find the autocorrelation function Rx(t,, t;), we have to evaluate E[X(t,)X(z;)], where 
X(t,) and_X(t,) are random variables obtained by observing the random process X(t) at times 
t, and t; respectively. 

Consider first the case when |t, — t;| > T. Under this condition the random variables 
X(t,) and X(t,) occur in different pulse intervals and are therefore independent. We thus have 


EIX (EAX) = E(X(4)JELX(t)] = 0, [evtl > T 


Consider next the case when | żę — t| < T, with t, = 0 and t; < tą. In such a situation 
we observe from Figure 1.6 that the random variables X(t,) and X(t;) occur in the same pulse 
interval if and only if the delay t, satisfies the condition ty < T — |t, — t;|. We thus obtain 
the conditional expectation: 

A, tas T— | - 6 


E[X(t,) X(t) lta] = i; cashes 


Averaging this result over all possible values of tz, we get 


T- | tet 


6 A’f Talta) dta 


T- |tt] Ar 
= [ T dt, 


= a(t | te = il), Ite t,| <T 


EIX = | 





T 


By similar reasoning for any other value of t}, we conclude that the autocorrelation function 
of a random binary wave, represented by the sample function shown in Figure 1.6, is only a 
function of the time difference T = tą — t; as shown by 


[zl 
Rx(s) = æfi - EN: aar (1.18) 
o, =T 


This result is plotted in Figure 1.7. a 
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FIGURE 1.7 Autocorrelation function of random binary wave. 


s CROSS-CORRELATION FUNCTIONS 


Consider next the more general case of two random processes X(t) and Y(t) with auto- 
correlation functions Rx(t, u) and Ry(t, u), respectively, The two cross-correlation func- 
tions of X(t) and Y(t) are defined by 


Rxy(t, u) = E[X(t)Y(w)] (1.19) 
and 
Ryx(t, u) = E[Y(#)X(v)] (1.20) 


where ¢ and u denote two values of time at which the processes are observed. In this case, 
the correlation properties of the two random processes X(t) and Y(t) may be displayed 
conveniently in matrix form as follows: 


Rx(t, u) Rxylt, aj 


ac ha u) Ryl u) 


which is called the correlation matrix of the random processes X(t) and Y(2). If the random 
processes X(t) and Y(t) are each stationary and, in addition, they are jointly stationary, 
then the correlation matrix can be written as 


_ |x?) Rxy(t) 
Rio) = |, | ak 


where t= t — u. 
The cross-correlation function is not generally an even function of 7 as was true for 
the autocorrelation function, nor does it have a maximum at the origin. However, it does 


‘obey a certain symmetry relationship as follows (see Problem 1.9): 


Rxy(t) = Ryx(—7) (1.22) 


b EXAMPLE 1.4 Quadrature-Modulated Processes 


Consider a pair of quadrature-modulated processes X,(t) and X(t) that are related to a sta- 
tionary process X(t) as follows: 

X(t) = X(t) cos(2rft + ©) 

X,(t) = X(t) sin(2rf,t + @) 
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where f, is a carrier frequency, and the random variable © is uniformly distributed over the 
interval [0, 27]. Moreover, ® is independent of X(t). One cross-correlation function of X,(t) 
and X,(t) is given by 

Ry2(t) = E[Xy(4)X2{t — 7)] 
E[X(#)X(t — 1) cos(2arf,t + ©) sin(2af,t — 27f.7 + ©) 
E[X()X(t — 7E[cos(2aft + ©) sin(2aft — 2nf.7r + O)] (1.23) 
$Rx(DE[sin(4 aft — 2af.r +20) — sin(27f.7)] 
—4Rx(7) sin(2af-7) 


i 


where, in the last line, we have made use of the uniform distribution of the random variable 
© representing phase. Note that at 7 = 0, the factor sin(27f,7) is zero and therefore 


Rı2(0) = EX, (#)X2(¢)] 
=0 


This shows that the random variables obtained by simultaneously observing the quadrature- 
modulated processes X(t) and X2(t) at some fixed value of time ¢ are orthogonal to each 
other. 


| 1.5 Ergodic Processes 


The expectations or ensemble averages of a random process X(t) are averages “across the 
process.” For example, the mean of a random process X(t) at some fixed time t, is the 
expectation of the random variable X(t,) that describes all possible values of the sample 
functions of the process observed at time t = t}. Naturally, we may also define long-term 
sample averages, or time averages that are averages “along the process.” We are therefore 
interested in relating ensemble averages to time averages, for time averages represent a 
practical means available to us for the estimation of ensemble averages of a random pro- 
cess. The key question, of course, is: When can we substitute time averages for ensemble 
averages? To explore this issue, consider the sample function x(t) of a stationary process 
X(t), with the observation interval defined as -T = t = T. The DC value of x(t) is defined 
by the time average 


1 F: 


Hx(T}) = ITJ” 


(t) dt (1.24) 


Clearly, the time average u,(T) is a random variable, as its value depends on the obser- 
vation interval and which particular sample function of the random process X(t) is picked 
for use in Equation (1.24). Since the process X(t) is assumed to be stationary, the mean of 
the time average u,(T) is given by (after interchanging the operation of expectation and 
integration): 


F. 
Elus(T)] =È | Elt] de 


1 [7 : 
= 57 |, ex at (1.25) 


= ux 
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where yy is the mean of the process X(t). Accordingly, the time average u,(T) represents 
an unbiased estimate of the ensemble-averaged mean uy. We say that the process X(t) is 
ergodic in the mean if two conditions are satisfied: 


> The time average u,(T) approaches the ensemble average ux in the limit as the 
observation interval T approaches infinity; that is, 


lim y,(T) = px 
T+2 


> The variance of u,(T), treated as a random variable, approaches zero in the limit as 
the observation interval T approaches infinity; that is, 
lim var[,(T)] = 0 
Tox 
The other time average of particular interest is the autocorrelation function R,(1, T) 
defined in terms of the sample function x(t) observed over the interval -T = t = T. 
Following Equation (1.24), we may formally define the time-averaged autocorrelation 
function of a sample function x(t) as follows: 


T 
R(t, T) == Oe x(t + r)x(t) dt (1.26) 


This second time-average should also be viewed as a random variable with a mean and 
variance of its own. In a manner similar to ergodicity of the mean, we say that the process 
x(t) is ergodic in the autocorrelation function if the following two limiting conditions are 
satisfied: 

lim R,(7, T) = Rx(7) 


T>% 


lim var[R.(T, T)] = 0 
Too 


We could, of course, go on in a similar way to define ergodicity in the most general 
sense by considering higher-order statistics of the process X(t). In practice, however, er- 
godicity in the mean and ergodicity in the autocorrelation function, as described here, are 
often (but not always) considered to be adequate. Note also that the use of Equations 
(1.24) and (1.26) to compute the time averages y,(T) and R,(¢, T) requires that the process 
X(t) be stationary. In other words, for a random process to be ergodic, it has to be sta- 
tionary; however, the converse is not necessarily true. 


1.6 Transmission of a Random Process 
Through a Linear Time-Invariant Filter 


Suppose that a random process X(t) is applied as input to a linear time-invariant filter of 
impulse response h(t), producing a new random process Y(t) at the filter output, as in 
Figure 1.8. In general, it is difficult to describe the probability distribution of the output 
random process Y(t), even when the probability distribution of the input random process 
X(t) is completely specified for -œ < ¢ < œ, 

In this section, we determine the time-domain form of the input-output relations of 
the filter for defining the mean and autocorrelation functions of the output random process 
¥(£) in terms of those of the input X(t), assuming that X(t) is a stationary process. 


1.6 Transmission of a Random Process Through a Linear Time-Invariant Filter 43 





impulse 
X() ——= response m> F() 
re) 











FicureE 1.8 Transmission of a random process through a linear time-invariant filter. 


The transmission of a process through a linear time-invariant filter is governed by 
the convolution integral; for a review of this operation, see Appendix 2. For the problem 
at hand, we may thus express the output random process Y(t) in terms of the input random 
process X(t) as 


Y(t) = T b(1)X(t — n) dr 


where 7; is the integration variable. Hence, the mean of Y(t) is 


By(t) = E[Y(t)] 


= al b(1,)X(t — 74) an] 


Provided that the expectation E[X(¢)] is finite for all t and the system is stable, we may 
interchange the order of expectation and integration in Equation (1.27) and so write 


(1.27) 


By(t) = fiz h(7,)E[X(t — %)] dr 
(1.28) 


= ie b(7)ux(t — n) dr 


When the input random process X(t) is stationary, the mean p(t) is a constant px, so 
that we may simplify Equation (1.28) as follows: 


Hy = Bx i: hin) dr 
= pxH(0) 


(1.29) 


where H(0) is the zero-frequency (DC) response of the system. Equation (1.29) states that 
the mean of the random process Y(t) produced at the output of a linear time-invariant 
system in response to X(t) acting as the input process is equal to the mean of X(t) multiplied 
by the DC response of the system, which is intuitively satisfying. 

Consider next the autocorrelation function of the output random process Y(t). By 
definition, we have 


Ry(t, u) = E[Y(¢)¥(w)] 
where ¢ and # denote two values of the time at which the output process is observed. We 


may therefore use the convolution integral to write 


Ry{t, u) = |f b(r,)X(t — n) dr, ia b(1)X(u — 1) dr {1.30) 
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Here again, provided that the mean-square value E[X°(£)] is finite for all t and the system 
is stable, we may interchange the order of the expectation and the integrations with respect 
to 7, and 7 in Equation (1.30), obtaining 


Rut = | dahin) | drobinJEIXte — 1)Xtu ~ nl 
è : (1.31) 
= K drh(r) f dth(m)Rx(t — m, u — m) 


When the input X(t) is a stationary process, the autocorrelation function of X(t) is only a 
function of the difference between the observation times t — 7, and u — 7. Thus, putting 
7 = t — u in Equation (1.31), we may write 


Ry(t) = E ee h(m)b(t)Rx(t — m + %) dt dr (1.32) 


On combining this result with that involving the mean py, we see that if the input to a 
stable linear time-invariant filter is a stationary process, then the output of the filter is also 
a stationary process. 

Since Ry(0) = E[Y*(z)], it follows that the mean-square value of the output random 
process Y(t) is obtained by putting 7 = 0 in Equation (1.32). We thus get the result 


eyo) = [fo birbindRen — n) dr dr (1.33) 


which is a constant. 


1.7 Power Spectral Density 


Thus far we have considered the characterization of stationary processes in linear systems 
in the time domain. We turn next to the characterization of random processes in linear 
systems by using frequency-domain ideas. In particular, we wish to derive the frequency- 
domain equivalent to the result of Equation (1.33) defining the mean-square value of the 
filter output. 

By definition, the impulse response of a linear time-invariant filter is equal to the 
inverse Fourier transform of the frequency response of the system; a review of the Fourier 
transform is presented in Appendix 2. Using H(f) to denote the frequency response of the 
system, we may thus write 


bia) = | Hf) expli2afn) df (1.34) 


Substituting this expression for (1) into Equation (1.33), we get 


ewan = | | | [HUN expat) af penata: - n) dr dry 
(1.35) 


=| ap nip) | dobre) [Rel — n) expli2afn) dr 
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In the last integral on the right-hand side of Equation (1.35), define a new variable 
TEN- N i 


Then we may rewrite Equation. (1.35) in the form 


RIY = f AfA [dubina explidafnn) | Rala) exp(-i2mfr) dr (1.36) 


However, the middle integral on the right-hand side in Equation (1.36) is simply H*(f), 
the complex conjugate of the frequency response of the filter, and so we may simplify this 
equation as 


m 


E[Y*(t)] = T df | H(f)|? T Rx(7) exp(—j2afr) dr (1.37) 


where |H(f)| is the magnitude response of the filter. We may further simplify Equation 
(1.37) by recognizing that the last integral is simply the Fourier transform of the auto- 
correlation function Rx(7) of the input random process X(t). This prompts us to introduce 
the definition of a new parameter 


Sx(f) = f. Rx(1) exp(—j2nfr) dr (1.38) 


The function Sx(f) is called the power spectral density, or power spectrum, of the station- 
ary process X(t). Thus substituting Equation (1.38) into (1.37), we obtain the desired 
relation: 


EO = [HUY I*Se(A) df (1.39) 


Equation (1.39) states that the mean-square value of the output of a stable linear time- 
invariant filter in response to a stationary process is equal to the integral over all frequen- 
cies of the power spectral density of the input process multiplied by the squared magnitude 
response of the filter. This is the desired frequency-domain equivalent to the time-domain 
relation of Equation (1.33). 

To investigate the physical significance of the power spectral density, suppose that 
the random process X(t) is passed through an ideal narrowband filter with a magnitude 
response centered about the frequency f,, as shown in Figure 1.9; that is, 


1, f+ fl <24f 


1.40 
0, [f+ fl>dar ee) 


IHA) =f 





Eyal: di 
Af Af 


FIGURE 1.9 Magnitude response of ideal narrowband filter. 


46 


CHAPTER 1 RANDOM PROCESSES 


where Af is the bandwidth of the filter. Then from Equation (1.39) we find that if the 
filter bandwidth Af is sufficiently small compared to the midband frequency f, and Sx(f) 
is a continuous function, the mean-square value of the filter output is approximately 


E[Y*(t)] = (2Af)Sx(f.) (1.41) 


The filter, however, passes only those frequency components of the input random process 
X(t) that lie inside a narrow frequency band of width Af centered about the frequency 
+f.. Thus Sx(f,) represents the frequency density of the average power in the random 
process X(t), evaluated at the frequency f = f.. The dimensions of the power spectral 
density are therefore in watts per Hertz (W/Hz). 


a PROPERTIES OF THE POWER SPECTRAL DENSITY 


The power spectral density Sx(f) and the autocorrelation function Rx(7) of a stationary 
process X(t) form a Fourier-transform pair with 7 and f as the variables of interest, as 
shown by the pair of relations 


œ 


Sx(f) = T Rx(7) exp(—j2afr) dr (1.42) 
Ral) = | | Self) expl af) df (1.43) 


Equations (1.42) and (1.43) are basic relations in the theory of spectral analysis of random 
processes, and together they constitute what are usually called the Einstein—Wiener- 
Khintchine relations.> 

The Einstein-Wiener-Khintchine relations show that if either the autocorrelation 
function or power spectral density of a random process is known, the other can be found 
exactly. But these functions display different aspects of the correlation information about 
the process, It is commonly accepted that for practical purposes, the power spectral density 
is the more useful “parameter.” 

We now wish to use this pair of relations to derive some general properties of the 
power spectral density of a stationary process. 


Property 1 


The zero-frequency value of the power spectral density of a stationary process equals the 
total area under the graph of the autocorrelation function; that is, 


Sx(0) = ia Rx(7) dr (1.44) 


This property follows directly from Equation (1.42) by putting f = 0. 


Property 2 


The mean-square value of a stationary process equals the total area under the graph of 
the power spectral density; that is, 


E[X?(t)] = C Sx(f) df (1.45) 


This property follows directly from Equation (1.43) by putting 7 = 0 and noting that 
Ry (0) = E[X*(2)]. 
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Property 3 
The power spectral density of a stationary process is always nonnegative; that is, 
Sx(f) = 0 for all f (1.46) 
This property is an immediate consequence of the fact that, in Equation (1.41), the 
mean-square value E[Y?(t)] must always be nonnegative. 
Property 4 


The power spectral density of a real-valued random process is an even function of fre- 
quency; that is, 


Sx(-f) = Sx(f) (1.47) 


This property is readily obtained by substituting —f for f in Equation (1.42): 
Sx(-f) = f Rx(7) exp(j2afr) dr 
Next, substituting —7 for 7, and recognizing that Ryx(—7) = Rx(7), we get 


Skif) = | Ria) exp(-j2afa) dr = Self) 


which is the desired result. 


Property 5 


The power spectral density, appropriately normalized, has the properties usually associated 
with a probability density function. 


The normalization we have in mind here is with respect to the total area under the 
graph of the power spectral density (i.e., the mean-square value of the process). Consider 
then the function 


px(f) = ==" Sx(f) (1.48) 
[sen ag 


In light of Properties 2 and 3, we note that px(f) = 0 for all f. Moreover, the total area 
under the function px(f) is unity. Hence, the normalized form of the power spectral den- 
sity, as defined in Equation (1.48), behaves similar to a probability density function. 


È ExamMpLe 1.5 Sinusoidal Wave with Random Phase (continued) 


Consider the random process X(t) = A cos(27f.t + ©), where @ is a uniformly distributed 
random variable over the interval [—7, 7]. The autocorrelation function of this random pro- 
cess is given by Equation (1.17), which is reproduced here for convenience: 
AZ 
Rx(7) = Tr cos(2. arf.) 
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FIGURE 1.10 Power spectral density of sine wave with random phase; 6(f) denotes the delta 
function at f = 0. 


Let 8(f) denote the delta function at f = 0; for the definition of the delta function and its 
properties, see Appendix 2. Taking the Fourier transform of both sides of the relation defining 
Rx(7), we find that the power spectral density of the sinusoidal process X(t) is 


Sx(f) = £ ESF — fe) + Of + fe) (1.49) 


which consists of a pair of delta functions weighted by the factor A?/4 and located at + f., as 
illustrated in Figure 1.10. We note that the total area under a delta function is one. Hence, 
the total area under Sy(f) is equal to A?/2, as expected. 4 


$ EXAMPLE 1.6 Random Binary Wave (continued) 


Consider again a random binary wave consisting of a sequence of 1s and Os represented by 
the values +A and —A, respectively. In Example 1.3 we showed that the autocorrelation 
function of this random process has a triangular waveform, as shown by 


Ral) = ae - tel), |r| <T 
0, |r| =T 
The power spectral density of the process is therefore 
Sx(f) = f as - 12) exp(—j2afr) dr 
Using the Fourier transform of a triangular function (see Table A6.3), we obtain 
Sx(f) = A°T sinc?(fT) (1.50) 


which is plotted in Figure 1.11. Here again we see that the power spectral density is nonneg- 
ative for all f and that it is an even function of f. Noting that Rx(0) = A? and using Property 
2, we find that the total area under Sx(f), or the average power of the random binary wave 
described here, is A”, which is intuitively satisfying. aq 


The result of Equation (1.50) may be generalized as follows. We note that the energy 


spectral density (i.e., the squared magnitude of the Fourier transform) of a rectangular 
pulse g(t) of amplitude .A and duration T is given by 


Elf) = AT? sinc*(fT) (1.51) 
We may therefore rewrite Equation (1.50) in terms of €,(f) simply as 
Elf) 
Sx(f) = = (1.52) 


T 
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FIGURE 1.11 Power spectral density of random binary wave. 


Equation (1.52) states that for a random binary wave in which binary symbols 1 and 0 
are represented by pulses g(t) and ~g(t), respectively, the power spectral density Sx(f) is 
equal to the energy spectral density €,(f) of the symbol shaping pulse g(t), divided by the 
symbol duration T. 


> ExamMece 1.7 Mixing of a Random Process with a Sinusoidal Process 


A situation that often arises in practice is that of mixing (i.e., multiplication) of a stationary 
process X(t) with a sinusoidal wave cos(27f,t + ©), where the phase @ is a random variable 
that is uniformly distributed over the interval [0, 27]. The addition of the random phase © 
in this manner merely recognizes the fact that the time origin is arbitrarily chosen when 
X(t) and cos(2 aft + ©) come from physically independent sources, as is usually the case. We 
are interested in determining the power spectral density of the random process Y(t), defined 
by 


Y(t) = X(t) cos(2af.t + O) (1.53) 


Using the definition of autocorrelation function of a stationary process and noting that the 
random variable © is independent of X(t), we find that the autocorrelation function of Y(t) is 
given by 
Ry(r) = E[Y(é + 7) Y(t)] 

= E[X(t + 7) cos(Qaf.t + Zafar + O)X(t) cos(2af,t + O)] 

= E[X(t + 7)X(t)]E[cos(2af,t + 2af.7 + O) cos(2af.t + O)] (1.54) 

= $Rx(7)E[cos(2rf,7) + cos(4af.t + 2xf.r + 20)] 

= $Rx(7) cos(2af.7) 
Because the power spectral density is the Fourier transform of the autocorrelation function, 


we find that the power spectral densities of the random processes X(t) and Y(t) are related as 
follows: 


Sy(f) = USx(f — fà + Sxlf + fA (1.55) 


According to Equation (1.55), the power spectral density of the random process Y(t) defined 
in Equation (1.53) is obtained as follows: We shift the given power spectral density Sx(f) of 
random process X(t) to the right by f., shift it to the left by f., add the two shifted power 
spectra, and divide the result by 4. í < 
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= RELATION AMONG THE POWER SPECTRAL DENSITIES 
OF THE INPUT AND OUTPUT RANDOM PROCESSES 


Let Sx(f) denote the power spectral density of the output random process Y(#) obtained 
by passing the random process X(t) through a linear filter of frequency response H(f). 
Then, recognizing by definition that the power spectral density of a random process is 
equal to the Fourier transform of its autocorrelation function and using Equation (1.32), 
we obtain 


Seip) = | Rola) expl—faaf) dr 
(1.56) 


= L T i h(7,)b()Rx(t — Tı + n) exp(—j2mfr) dr dr, dr 


Let T — 7, + T2 = To, or, equivalently, T = To + 7, — Ta. Then by making this substitution 
in Equation (1.56), we find that Sx(f) may be expressed as the product of three terms: the 
frequency response H(f) of the filter, the complex conjugate of H(f), and the power spec- 
tral density Sx(f) of the input random process X(t). We may thus simplify Equation (1.56) 
as 


Sy(f) = HA(f)H*(f)Sx(f) (1.57) 


Finally, since | H(f) |? = H(f)H* (f), we find that the relationship among the power spectral 
densities of the input and output random processes is expressed in the frequency domain 
by writing 


Sy(f) = |H) Sxl) (1.58) 


Equation (1.58) states that the power spectral density of the output process Y(t) equals 
the power spectral density of the input process X(t) multiplied by the squared magnitude 
response of the filter. By using this relation, we can therefore determine the effect of passing 
a random process through a stable, linear, time-invariant, filter. In computational terms, 
Equation (1.58) is usually easier to handle than its time-domain counterpart of Equation 
(1.32), involving the autocorrelation function. 


= RELATION AMONG THE POWER SPECTRAL DENSITY 
AND THE MAGNITUDE SPECTRUM OF A SAMPLE FUNCTION 


We now wish to relate the power spectral density Sx(f) directly to the spectral properties 
of a sample function x(t) of a stationary process X(t) that is ergodic. For the sample 
function x(t) to be Fourier transformable, however, it must be absolutely integrable; that 
is : 


E |x(t)| dt < œ (1.59) 


This condition can never be satisfied by any stationary sample function x(t) of infinite 
duration. In order to use the Fourier transform technique, we consider a truncated segment 
of x(t), defined over the observation interval -T = t < T, say. Thus, using X(f, T) to 
denote the Fourier transform of the truncated sample function so defined, we may write 


F 


X(f, T) = i. x(t) exp(—j2aft) dt (1.60) 
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Assuming that the process x(t) is also ergodic, we may evaluate the autocorrelation 
function Rx(7) of X(t) using the time-average formula (see Section 1.5) 
1 
Rx(7) = lim = pile + 7)x(t) dt (1.61) 
T> a 
It is customary to view the sample function x(t) as a power signal (i.e., a signal with finite 
average power). Hence, we may formulate the following Fourier-transform pair: 
T 


1 
zg | ME + Male) dt = z lX D (1.62) 


The parameter on the left-hand side is a time-averaged autocorrelation function. The pa- 
rameter on the right-hand side is called the periodogram, whose dimensions are the same 
as those of the power spectral density. This terminology is a misnomer, however, since the 
periodogram is a function of frequency, not period. Nevertheless, it has wide usage. The 
quantity was first used by statisticians to look for periodicities such as seasonal trends in 
data. 

Using the formula for the inverse Fourier transform in the Fourier-transform pair of 
Equation (1.62), we may express the time-averaged autocorrelation function of the sample 
function x(t) in terms of the periodogram as 


17 a! , 
oT k x(t + t)x(t) dt = k oT |X(f, T)|* exp(j2mfT) df (1.63) 
Hence, substituting Equation (1.63) into (1.61), we get 
Rx(r) = lim L |X(f, T)|? exp(j2afr) df (1.64) 
Tox J-» 2T 


For a fixed value of the frequency f, the periodogram is a random variable in that 
its value varies in a random manner from one sample function of the random process to 
another. Thus, for a given sample function x(t), the periodogram does not converge in any 
statistical sense to a limiting value as T tends to infinity. As such, it would be incorrect to 
interchange the order of the integration and limiting operations in Equation (1.64). Sup- 
pose, however, that we take the expectation of both sides of Equation (1.64) over the 
ensemble of all sample functions of the random process and recognize that for an ergodic 
process the autocorrelation function Rx(7) is unchanged by such an operation. Then, since 
each sample function of an ergodic process eventually takes on nearly all the modes of 
behavior of each other sample function, we may thus write 


æ 


Rx(a) = lim | z EIXU, T)|?] expli2mfn) df (1.65) 


T=>œ ~o 


Now we may interchange the order of the integration and limiting operations and so obtain 


Rain) = [7 {tim 2 EIXU DPI) expli2ape df (1.66) 


Hence, comparing Equations (1.66) and (1.43), we obtain the desired relation between 
the power spectral density S,(f) of an ergodic process and the squared magnitude spectrum 
|X(f, T)|? of a truncated sample function of the process: 


1 
Sx(f) = lim zy ELX(f, 7) "I 
os (1.67) 


] 





T 
lim A e| fi x(t) exp(—j2rft) dt 


Tope dE: 
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It is important to note that in Equation (1.67) it is not possible to let T — œ before taking 
the expectation. Equation (1.67) provides the mathematical basis for estimating‘ the power 
spectral density of an ergodic random process, given a sample function x(t) of the process 
observed over the interval [—T, T]. 


= Cross-SPECTRAL DENSITIES 


Just as the power spectral density provides a measure of the frequency distribution of a 
single random process, cross-spectral densities provide a measure of the frequency inter- 
relationship between two random processes. In particular, let X(t) and Y(t) be two jointly 
stationary processes with their cross-correlation functions denoted by Rxy(7) and Ryx(1). 
We then define the cross-spectral densities Sxy(f) and Syx(f) of this pair of random pro- 
cesses to be the Fourier transforms of their respective cross-correlation functions, as 
shown by 


Sxy(f) = Ja Rxy(7) exp(—j2rfr) dr (1.68) 


and 


Syx(f) = ie Ryx(7) exp(—j2afr) dr (1.69) 


The cross-correlation functions and cross-spectral densities thus form Fourier-transform 
pairs. Accordingly, using the formula for inverse Fourier transformation we may also 
write 


Rxy(7) = B Sxy(f) expl j2rfr) df (1.70) 
and 


Ryx(7) = E Syx(f) explj2rfr) df (1.71) 


The cross-spectral densities Syy(f) and Syx(f) are not necessarily real functions of 
the frequency f. However, substituting the relationship 


Ryy(7) = Ryx(—7) 


into Equation (1.68) and then using Equation (1.69) we find that Sxy(f) and Syx(f) are 
related by 


Sxy(f) = Syx(-f) = Syx(f) (1.72) 


& EXAMPLE 1.8 


Suppose that the random processes X(t) and Y(t) have zero mean, and they are individually 
stationary. Consider the sum random process 


Z(t) = X(t) + Y(t) 
The problem is to determine the power spectral density of Z(t). 
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The autocorrelation function of Z(£) is given by 
Rz(t, u) = E[Z(t)Z(u)] 
= El(X(t) + YHX) + Y(w))] 
= E[X(e)X(w)] + EIXE) Y(u)] + ELY(@)X(w)] + ELY Y(u)] 
= Rx(t, u) + Reylt, u) + Ryx(t, u) + Rylt, u) 
Defining 7 = t — u, we may therefore write 
R27) = Rx(z7) + Rxy(t) + Ryx(7) + Ry(7) (1.73) 


when the random processes X(t) and Y(t) are also jointly stationary. Accordingly, taking the 
Fourier transform of both sides of Equation (1.73), we get 


SAF) = Salf) + Sxv(f) + Syxl(f) + Sy(f) (1.74) 


We thus see that the cross-spectral densities Sxy(f) and Syx(f) represent the spectral compo- 
nents that must be added to the individual power spectral densities of a pair of correlated 
random processes in order to obtain the power spectral density of their sum. 

When the stationary processes X(t) and Y(z) are uncorrelated, the cross-spectral densities 
Sxy(f) and Syx(f) are zero, and so Equation (1.74) reduces as follows: 


Sulf) = Sx(f) + Sy(f) (1.75) 


We may generalize this latter result by stating that when there is a multiplicity of zero-mean 
stationary processes that are uncorrelated with each other, the power spectral density of their 
sum is equal to the sum of their individual power spectral densities. 


b> EXAMPLE 1.9 


Consider next the problem of passing two jointly stationary processes through a pair of sep- 
arate, stable, linear, time-invariant filters, as shown in Figure 1.12. In particular, suppose that 
the random process X(t) is the input to the filter of impulse response /,(¢) and that the random 
process Y(f) is the input to the filter of impulse response b(t). Let V(t) and Z(t) denote the 
random processes at the respective filter outputs. The cross-correlation function of V(t) and 
Z(t) is therefore 


Ryzlt, u) = E[V(A)Z(x)] 


= elf hi(m4)X(t — n) dr in hy(m)¥(u — n) an] 


om a (1.76) 
= a i hi(t)hol m)E[X(t — 1) ¥(u@ — n)] dn dr 


= i i hilt)holt)Rxvlt — T, u — n) dr drz 
where Rxy(#, u) is the cross-correlation function of X(t) and Y(t). Because the input random 
processes are jointly stationary (by hypothesis), we may set 7 = t — # and so rewrite Equation 


(1.76) as follows: 


Ryz(t) = a 1 hy(1)bo(t2)Ray(t — 11 + n) dr drz (1.77) 


w Me ee YQ) — h LZ) 


FIGURE 1.12 A pair of separate linear time-invariant filters. 
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Taking the Fourier transform of both sides of Equation (1.77) and using a procedure 
similar to that which led to the development of Equation (1.39), we finally get 


Sv2lf) = Hi(f)H(f)Sxxtf) (1.78) 


where H,(f) and H2(f) are the frequency responses of the respective filters in Figure 1.12, and 
HÌ} (f) is the complex conjugate of H2(f). This is the desired relationship between the cross- 
spectral density of the output processes and that of the input processes. 4 


i 1.8 Gaussian Process 


The material we have presented on random processes up to this point in the discussion 
has been of a fairly general nature. In this section, we consider an important family of 
random processes known as Gaussian processes.* 

Let us suppose that we observe a random process X(t) for an interval that starts at 
time t = 0 and lasts until t = T. Suppose also that we weight the random process X(t) by 
some function g(t) and then integrate the product g(t)X(t) over this observation interval, 
thereby obtaining a random variable Y defined by 


Y =| g(t)X(t) dt (1.79) 


We refer to Y as a linear functional of X(t). The distinction between a function and a 
functional should be carefully noted. For example, the sum Y = =“.,a;X;, where the a; are 
constants and the X; are random variables, is a linear function of the X,; for each observed 
set of values for the random variables X;, we have a corresponding value for the random 
variable Y. On the other hand, in Equation (1.79) the value of the random variable Y 
depends on the course of the argument function g(t)X(t) over the entire observation in- 
terval from 0 to T. Thus a functional is a quantity that depends on the entire course of 
one or more functions rather than on a number of discrete variables. In other words, the 
domain of a functional is a set or space of admissible functions rather than a region of a 
coordinate space. 

If in Equation (1.79) the weighting function g(t) is such that the mean-square value 
of the random variable Y is finite, and if the random variable Y is a Gaussian-distributed 
random variable for every g(t) in this class of functions, then the process X(t) is said to be 
a Gaussian process. In other words, the process X(t) is a Gaussian process if every linear 
functional of X(t) is a Gaussian random variable. 

We say that the random variable Y has a Gaussian distribution if its probability 
density function has the form 





1 (y — py)? 
fey) = Vins “l ee | (1.80) 


where puy is the mean and o¥ is the variance of the random variable Y. A plot of this 
probability density function is given in Figure 1.13 for the special case when the Gaussian 
random variable Y is normalized to have a mean py of zero and a variance ø% of one, as 
shown by 


Boo eed, 


Such a normalized Gaussian distribution is commonly written as N(0, 1). 
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FIGURE 1.13 Normalized Gaussian distribution. 


A Gaussian process has two main virtues. First, the Gaussian process has many 
properties that make analytic results possible; we will discuss these properties later in the 
section. Second, the random processes produced by physical phenomena are often such 
that a Gaussian model is appropriate. Furthermore, the use of a Gaussian model to describe 
the physical phenomena is usually confirmed by experiments. Thus the frequent occurrence 
of physical phenomena for which a Gaussian model is appropriate, together with the ease 
with which a Gaussian process is handled mathematically, make the Gaussian process very 
important in the study of communication systems. 


= CENTRAL Limit THEOREM 


The central limit theorem provides the mathematical justification for using a Gaussian 
process as a model for a large number of different physical phenomena in which the 
observed random variable, at a particular instant of time, is the result of a large number 
of individual random events. To formulate this important theorem, let X, i = 1, 2,..., 
N, be a set of random variables that satisfies the following requirements: 


1. The X; are statistically independent. 
2. The X; have the same probability distribution with mean wx and variance o%. 


The X; so described are said to constitute a set of independently and identically distributed 
(i.i.d.) random variables. Let these random variables be normalized as follows: 


Ve wag. 220 Soh 
Tx 


so that we have 


EY] = 0 
and 
var[Y,] = 1 
Define the random variable 
1 


Vu = SS 


Y, 
VN? 


iM z 
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The central limit theorem states that the probability distribution of Vx approaches a nor- 
malized Gaussian distribution N(0, 1) in the limit as the number of random variables N 
approaches infinity. 

It is important to realize, however, that the central limit theorem gives only the 
“limiting” form of the probability distribution of the normalized random variable Vy as 
N approaches infinity. When N is finite, it is sometimes found that the Gaussian limit gives 
a relatively poor approximation for the actual probability distribution of Vyn even though 
N may be quite large. 


PROPERTIES OF A GAUSSIAN PROCESS 


A Gaussian process has some useful properties that are described in the sequel. 


Property 1 
If a Gaussian process X(t) is applied to a stable linear filter, then the random process Y(t) 
developed at the output of the filter is also Gaussian. 


This property is readily derived by using the definition of a Gaussian process based 
on Equation (1.79). Consider the situation depicted in Figure 1.8, where we have a linear 
time-invariant filter of impulse response h(t), with the random process X(t) as input and 
the random process Y(t) as output. We assume that X(t) is a Gaussian process. The random 
processes Y(t) and X(t) are related by the convolution integral 


n= | we- ax 1) dr, O<t<o (1.81) 


We assume that the impulse response h(t) is such that the mean-square value of the output 
random process Y(t) is finite for all ż in the range 0 = t < œ for which Y(t) is defined. To 
demonstrate that the output process Y(t) is Gaussian, we must show that any linear func- 
tional of it is a Gaussian random variable. That is, if we define the random variable 


Z= l gto f b(t — 7)X(7) dr dt (1.82) 


then Z must be a Gaussian random variable for every function gy(t), such that the mean- 
square value of Z is finite. Interchanging the order of integration in Equation (1.82), we 
get 


Z= f, sox Xid (1.83) 


where 
= iE: gyl(t)b(t — r) dr (1.84) 


Since X(t) is a Gaussian process by hypothesis, it follows from Equation (1.83) that Z 
must be a Gaussian random variable. We have thus shown that if the input X() to a linear 
filter is a Gaussian process, then the output Y(t) is also a Gaussian process. Note, however, 
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that although our proof was carried out assuming a time-invariant linear filter, this prop- 
erty is true for any arbitrary stable linear system. 


Property 2 

Consider the set of random variables or samples X(t,), X(t2),..., X(t,), obtained by 
observing a random process X(t) at times ty, tr,..., ta If the process X(t) is Gaussian, 
then this set of random variables is jointly Gaussian for any n, with their n-fold joint 
probability density function being completely determined by specifying the set of means 


Mx) = E[X(t)], 1=1,2,...,” 


and the set of covariance functions 
Cx(te, t) = E[(X(t,) Fä Exin (X(t) = Exe), k, i= 1, 2, sory A 


Let the #-by-1 vector X denote the set of random variables X(t,),..., X(t,) derived from 
the Gaussian process X(t) by sampling it at times t,,..., t. Let x denote a value of X. 
According to Property 2, the random vector X has a muttivariate Gaussian distribution 
defined in matrix form as 








1 1 
Fe, ae XXi seg x2) (2m2 a2 exo 2 (x u) (x = »») (1.85) 


where the superscript T denotes transposition and 
p = mean vector 


= T 
= [Has Mass + +s Mnl 

> = covariance matrix 

{Cx(te, ti) ier 

X1 = inverse of covariance matrix 


A = determinant of covariance matrix ¥ 


Property 2 is frequently used as the definition of a Gaussian process. However, this 
definition is more difficult to use than that based on Equation (1.79) for evaluating the 
effects of filtering on a Gaussian process. 

We may extend Property 2 to two (or more) random processes as follows, Consider 
the composite set of random variables X(t,), X(t),..., X(t,), Y(u1), Y(#2),- ~~, Yun) 
obtained by observing a random process X(t) at times {t; i = 1,2,..., n}, and a second 
random process Y(t) at times {u}, k = 1, 2,..., m}. We say that the processes X(t) and 
Y(t) are jointly Gaussian if this composite set of random variables is jointly Gaussian for 
any # and m. Note that in addition to the mean and correlation functions of the random 
processes X(t) and Y(t) individually, we must also know the cross-covariance function 


El(X(t) — Hx (Y(t) — Myee)] = Rxvltis te) — Mxteab yen) 


for any pair of observation instants (t;, #4). This additional knowledge is embodied in the 
cross-correlation function, Ryy(t;, 4,), of the two processes X(t) and Y(t). 


Property 3 
If a Gaussian process is stationary, then the process is also strictly stationary. 


This follows directly from Property 2. 
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Property 4 


If the random variables X(t,), X(t), . . . , X(t,), obtained by sampling a Gaussian process 
X(t) at times ty, ta, ..., t,, are uncorrelated, that is, 


El(X(t) — axe (Xt) — exe] = 0, it#k 
then these random variables are statistically independent. 


The uncorrelatedness of X(t,),..., X(t,,) means that the covariance matrix ¥ is a 
diagonal matrix as shown by 


ot O 


where ` 
o? = ENX) - E[X(t))", i= 1,2,...50 


Under this condition, the multivariate Gaussian distribution of Equation (1.85) simplifies 
to 


fx(x) = IT fx{xi) 


where X; = X(t,) and 


c 1 - (x; ca ux) 
fx{xi) zá VITO; ox 207 


In words, if the Gaussian random variables X(t;),..., X(#,) are uncorrelated, then they 
are statistically independent, which, in turn, means that the joint probability density func- 
tion of this set of random variables can be expressed as the product of the probability 
density functions of the individual random variables in the set. 


i 1.9 Noise 


The term noise is used customarily to designate unwanted signals that tend to disturb the 
transmission and processing of signals in communication systems and over which we have 
incomplete control. In practice, we find that there are many potential sources of noise in 
a communication system. The sources of noise may be external to the system (e.g., at- 
mospheric noise, galactic noise, man-made noise), or internal to the system. The second 
category includes an important type of noise that arises from spontaneous fluctuations of 
current or voltage in electrical circuits.® This type of noise represents a basic limitation on 
the transmission or detection of signals in communication systems involving the use of 
electronic devices. The two most common examples of spontaneous fluctuations in elec- 
trical circuits are shot noise and thermal noise, which are described in the sequel. 


E SHOT NOISE 


Shot noise arises in electronic devices such as diodes and transistors because of the discrete 
nature of current flow in these devices. For example, in a photodetector circuit a current 
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pulse is generated every time an electron is emitted by the cathode due to incident light 
from a source of constant intensity. The electrons are naturally emitted at random times 
denoted by T}, where ~œ < k < œ, It is assumed that the random emissions of electrons 
have been going on for a long time. Thus, the total current flowing through the photo- 
detector may be modeled as an infinite sum of current pulses, as shown by 


X= X bt- x) (1.86) 
k=—o 
where h(t — 7) is the current pulse generated at time 7. The process X(t) defined by 
Equation (1.86) is a stationary process called shot noise. 

The number of electrons, N(t), emitted in the time interval [0, £] constitutes a discrete 
stochastic process, the value of which increases by one each time an electron is emitted. 
Figure 1.14 shows a sample function of such a process. Let the mean value of the number 
of electrons, v, emitted between times ¢ and £ + to be defined by 


E[v] = Ato (1.87) 


The parameter A is a constant called the rate of the process. The total number of electrons 
emitted in the interval [ż, £ + tg], that is, 


v= N(t + to) — N(t) 


follows a Poisson distribution with a mean value equal to Ato. In particular, the probability 
that k electrons are emitted in the interval [t, t + to] is defined by 


k 
er Wa)” ge k=0, 1... (1.88) 


Unfortunately, a detailed statistical characterization of the shot-noise process X(t) 
defined in Fquation (1.86) is a difficult mathematical task. Here we simply quote the results 
pertaining to the first two moments of the process: 


» The mean of X(t) is 
px =A T h(t) dt (1.89) 


where A is the rate of the process and h(t) is the waveform of a current pulse. 
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Ficure 1.14 Sample function of a Poisson counting process. 


60 


CHAPTER 1 8 RANDOM PROCESSES 


> The autocovariance function of X(t) is 
Cxi =A j b(t)h(t + 7) dt (1.90) 


This second result is known as Campbell’s theorem. 


For the special case of a waveform k(t) consisting of a rectangular pulse of amplitude 
A and duration T, the mean of the shot-noise process X(t) is AAT, and its autocovariance 
function is 


capa PRT = lel, I< 
x(t) = 0, |r| =T 


which has a triangular form similar to that shown in Figure 1.7. 


THERMAL NOISE 


Thermal noise is the name given to the electrical noise arising from the random motion of 
electrons in a conductor. The mean-square value of the thermal noise voltage Vry ap- 
pearing across the terminals of a resistor, measured in a bandwidth of Af Hertz, is, for all 
practical purposes, given by 


E[Vin] = 4kTR Af volts? (1.91) 


where k is Boltzmann’s constant equal to 1.38 X 107% joules per degree Kelvin, T is the 
absolute temperature in degrees Kelvin, and R is the resistance in ohms. We may thus 
model a noisy resistor by the Thévenin equivalent circuit consisting of a noise voltage 
generator of mean-square value E[V7,] in series with a noiseless resistor, as in Figure 
1.154. Alternatively, we may use the Norton equivalent circuit consisting of a noise current 
generator in parallel with a noiseless conductance, as in Figure 1.156. The mean-square 
value of the noise current generator is 


1 
EHn] = +3 ELVAN 
[ =n] R2 [ ] ( 1.92) 
= 4kTG Af amps? 
where G = 1/R is the conductance. It is also of interest to note that because the number 
of electrons in a resistor is very large and their random motions inside the resistor are 


statistically independent of each other, the central limit theorem indicates that thermal 
noise is Gaussian distributed with zero mean. 


ALI G 


Elvin 


(a) (b) 


FIGURE 1.15 Models of a noisy resistor. (a) Thévenin equivalent circuit. (b) Norton equivalent 
circuit. 
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Noise calculations involve the transfer of power, and so we find that the use of the 
maximum-power transfer theorem is applicable to such calculations. This theorem states 
that the maximum possible power is transferred from a source of internal resistance R to 
a load of resistance R; when R; = R. Under this matched condition, the power produced 
by the source is divided equally between the internal resistance of the source and the load 
resistance, and the power delivered to the load is referred to as the available power. Ap- 
plying the maximum-power transfer theorem to the Thévenin equivalent circuit of Figure 
1.154 or the Norton equivalent circuit of Figure 1.15), we find that a noisy resistor pro- 
duces an available noise power equal to kT Af watts. 


æ WHITE NOISE 


The noise analysis of communication systems is customarily based on an idealized form 
of noise called white noise, the power spectral density of which is independent of the 
operating frequency. The adjective white is used in the sense that white light contains equal 
amounts of all frequencies within the visible band of electromagnetic radiation. We express 
the power spectral density of white noise, with a sample function denoted by w(t), as 


N. 
Sw(f) = S (1.93) 


which is illustrated in Figure 1.164. The dimensions of No are in watts per Hertz. The 
parameter No is usually referenced to the input stage of the receiver of a communication 
system. It may be expressed as 


N, = kT, (1.94) 


where k is Boltzmann’s constant and T, is the equivalent noise temperature of the receiver.” 
The equivalent noise temperature of a system is defined as the temperature at which a 
noisy resistor has to be maintained such that, by connecting the resistor to the input of a 
noiseless version of the system, it produces the same available noise power at the output 
of the system as that produced by all the sources of noise in the actual system. The im- 
portant feature of the equivalent noise temperature is that it depends only on the param- 
eters of the system. 

Since the autocorrelation function is the inverse Fourier transform of the power 
spectral density, it follows that for white noise 


=M 


Ry{7) > 


6(7) (1.95) 
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Figure 1.16 Characteristics of white noise. (a) Power spectral density. (b) Autocorrelation 
function. 
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That is, the autocorrelation function of white noise consists of a delta function weighted 
by the factor No/2 and occurring at 7 = 0, as in Figure 1.165. We note that R w(7) is zero 
for t # 0. Accordingly, any two different samples of white noise, no matter how closely 
together in time they are taken, are uncorrelated. If the white noise w(t) is also Gaussian, 
then the two samples are statistically independent. In a sense, white Gaussian noise rep- 
resents the ultimate in “randomness.” 

Strictly speaking, white noise has infinite average power and, as such, it is not phys- 
ically realizable. Nevertheless, white noise has simple mathematical properties exemplified 
by Equations (1.93) and (1.95), which make it useful in statistical system analysis. 

The utility of a white noise process is parallel to that of an impulse function or delta 
function in the analysis of linear systems. Just as we may observe the effect of an impulse 
only after it has been passed through a system with a finite bandwidth, so it is with white 
noise whose effect is observed only after passing through a similar system. We may state, 
therefore, that as long as the bandwidth of a noise process at the input of a system is 
appreciably larger than that of the system itself, then we may model the noise process as 
white noise. 


> EXAMPLE 1.10 Ideal Low-Pass Filtered White Noise 


Suppose that a white Gaussian noise w(t) of zero mean and power spectral density No/2 is 
applied to an ideal low-pass filter of bandwidth B and passband magnitude response of one. 
The power spectral density of the noise z(t) appearing at the filter output is therefore (see 


Figure 1.17) 
No | 
Snif) =42°> ~B<F<8 (1.96) 
0, |f| >B 


The autocorrelation function of n(t) is the inverse Fourier transform of the power spectral 
density shown in Figure 1.17: 


B 


Rair) = | | explizate) df o 


= NoB sinc(2B7) 


This autocorrelation function is plotted in Figure 1.17b. We see that Ry(7) has its maximum 
value of NoB at the origin, and it passes through zero at 7 = +k/2B, where k = 1, 2, 3,+--. 
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FIGURE 1.17 Characteristics of low-pass filtered white noise. (2) Power spectral density. (b) Auto- 
correlation function, 
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Since the input noise w(t) is Gaussian (by hypothesis), it follows that the band-limited 
noise 7(t) at the filter output is also Gaussian. Suppose now that x(t) is sampled at the rate of 
2B times per second. From Figure 1.17b, we see that the resulting noise samples are uncor- 
related and, being Gaussian, they are statistically independent. Accordingly, the joint proba- 
bility density function of a set of noise samples obtained in this way is equal to the product 
of the individual probability density functions. Note that each such noise sample has a mean 
of zero and variance of NoB. Ei 


= EXAMPLE 1.11 Correlation of White Noise with a Sinusoidal Wave 


Consider the sample function 


B (T 
w'(t) = EI w(t) cos(27f.t) dt (1.98) 


which is the output of a correlator with white Gaussian noise w(t) and sinusoidal wave 
V2/T cos(27f.t) as inputs; the scaling factor V2/T is included here to make the sinusoidal 
wave input have unit energy over the interval 0 = ż = T. (This problem was encountered in 
the Background and Preview chapter but was not elaborated on at that time.) With the noise 
w(t) having zero mean, it immediately follows that the correlator output w’(t) has zero mean, 
too. The variance of the correlator output is defined by 


2 T (T 
P =E Bi [ w(t,) cos(2zf.t;)w(t2) cos(2arf.t2) dt, d| 
=2 f f Etot t Irfa aftr) dt, d 
=F Jo h w(t,)w(t2)] cos(2aft1) cos(2rf.t2) dt, dtz 


T T 
= a f Rwdti, th) cos(27f,t,) cos(2mftz) dt, dt, 


where Ry(t;, t2) is the autocorrelation function of the white noise w(t). But from Equation 
(1.95): 


N, 
Rwlts b) = a A(t, — t) 


where No/2 is the power spectral density of the white noise w(t). Accordingly, we may simplify 
the expression for the variance g? as 
No 2" f* 
= S sT $ Í 8(t, — t) cos(2arf;tı) cos(2mf.t,) dt, dt, 
We now invoke the sifting property of the delta function, namely, 


I g(t) ôlt) dt = g(0) 


where g(t) is a continuous function of time, assuming the value g(0) at time t = 0. Hence, we 
may further simplify o” as 
No 2 {7 
o = -=| cos*(2af.t) dt 
2 Th (aft) (1.99) 
No 
2 


where it is assumed that the frequency f, of the sinusoidal wave input is an integer multiple 
of the reciprocal of T. 
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FIGURE 1.18 (a) Power spectral density of narrowband noise. (b) Sample function of narrow- 
band noise. 


i 1.10 Narrowband Noise 


The receiver of a communication system usually includes some provision for preprocessing 
the received signal. The preprocessing may take the form of a narrowband filter whose 
bandwidth is just large enough to pass the modulated component of the received signal 
essentially undistorted but not so large as to admit excessive noise through the receiver. 
The noise process appearing at the output of such a filter is called narrowband noise. With 
the spectral components of narrowband noise concentrated about some midband fre- 
quency +f, as in Figure 1.182, we find that a sample function n(t) of such a process appears 
somewhat similar to a sine wave of frequency f., which undulates slowly in both amplitude 
and phase, as illustrated in Figure 1.18). 

To analyze the effects of narrowband noise on the performance of a communication 
system, we need a mathematical representation of it. Depending on the application of 
interest, there are two specific representations of narrowband noise: 


1. The narrowband noise is defined in terms of a pair of components called the in-phase 
and quadrature components. 

2. The narrowband noise is defined in terms of two other components called the en- 
velope and phase. 


These two representations are described in what follows. For now it suffices to say that 
given the in-phase and quadrature components, we may determine the envelope and phase 
components, and vice versa: Moreover, in their own individual ways, the two represen- 
tations are not only basic to the noise analysis of communication systems but also to the 
characterization of narrowband noise itself. 


1.11 Representation of Narrowband Noise 
in Terms of In-Phase and Quadrature Components 





Consider a narrowband noise n(t) of bandwidth 2B centered on frequency f,, as illustrated 
in Figure 1.18. In light of the theory of band-pass signals and systems presented in Ap- 
pendix 2, we may represent z(t) in the canonical (standard) form: 


nit) = n,(t) cos(2mf.t) — no(t) sin(2af,t) (1.100) 
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where x,(t) is called the in-phase component of n(t), and no(t) is called the quadrature 
component of n(t). Both m,(¢) and no(t) are low-pass signals. Except for the midband 
frequency f,, these two components are fully representative of the narrowband noise n(t). 

Given the narrowband noise x(t), we may extract its in-phase and quadrature com- 
ponents using the scheme shown in Figure 1.194. It is assumed that the two low-pass filters 
used in this scheme are ideal, each having a bandwidth equal to B (i.e., one-half the band- 
width of the narrowband noise x(t)). The scheme of Figure 1.194 follows from the rep- 
resentation of Equation (1.100). We may, of course, use this equation directly to generate 
the narrowband noise x(t), given its in-phase and quadrature components, as shown in 
Figure 1.196. The schemes of Figures 1.194 and 1.19b may thus be viewed as narrowband 
noise analyzer and synthesizer, respectively. 

The in-phase and quadrature components of a narrowband noise have important 
properties that are summarized here: 


1. The in-phase component m,(t) and quadrature component no(t) of narrowband noise 
n(t) have zero mean. 

2. If the narrowband noise m(t) is Gaussian, then its in-phase component n;(t) and 
quadrature component no(t) are jointly Gaussian. 

3. If the narrowband noise x(t) is stationary, then its in-phase component n,(t) and 
quadrature component no(t) are jointly stationary. 

4. Both the in-phase component #,(t) and quadrature component no(t) have the same 
power spectral density, which is related to the power spectral density Sy{f) of the 
narrowband noise n(t) as 


Su lf) = Snol f) = { 


where it is assumed that Sy(f) occupies the frequency interval f. — B < | f| = f, + B, 
and f, > B. 

5. The in-phase component #,(t) and quadrature component no(t) have the same vari- 
ance as the narrowband noise n(t). 

6. The cross-spectral density of the in-phase and quadrature components of narrow- 
band noise x(t) is purely imaginary, as shown by 


Snif = f) + Sf +f), —Bsf=sB 


1.101 
0, otherwise ( ) 


Suinglf) = Sunon f) (1.102) 
= ad +f) -= Sif - fA, -Bsf=sB 
0, otherwise 
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FıGuRe 1.19 (a) Extraction of in-phase and quadrature components of a narrowband process. 
(b) Generation of a narrowband process from its in-phase and quadrature components, 
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7, If the narrowband noise v(t) is Gaussian and its power spectral density S(t) is sym- 
metric about the mid-band frequency f., then the in-phase component #;(t) and 
quadrature component no(t) are statistically independent. 


For further discussions of these properties, the reader is referred to Problems 1.28 and 
1.29. 


> EXAMPLE 1.12 Ideal Band-Pass Filtered White Noise 


Consider a white Gaussian noise of zero mean and power spectral density No/2, which is 
passed through an ideal band-pass filter of passband magnitude response equal to one, mid- 
band frequency f., and bandwidth 2B. The power spectral density characteristic of the filtered 
noise 1(t) will therefore be as shown in Figure 1.204. The problem is to determine the auto- 
correlation functions of n(t) and its in-phase and quadrature components. 

The autocorrelation function of z(t) is the inverse Fourier transform of the power spec- 
tral density characteristic shown in Figure 1.20a: 


—f_tB No : ftB No : 
Rael = f, SP expia af + Jig Eein af 
= NoB sinc(2Br)[exp(—j2rf;T) + exp(j27f.7)] (1.103) 
= 2NoB sinc(2B7) cos(27f.7) 
which is plotted in Figure 1.20b. 


The spectral density characteristic of Figure 1.204 is symmetric about +f.. Therefore, 
: we find that the corresponding spectral density characteristic of the in-phase noise component 
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FIGURE 1.20 Characteristics of ideal band-pass filtered white noise. (2) Power spectral density. 
(b) Autocorrelation function. (c) Power spectral density of in-phase and quadrature components. 
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n,(t) or the quadrature noise component no(t) is as shown in Figure 1.21c. The autocorrelation 
function of n;{ż} or no(t) is therefore (see Example 1.10): 


Rut) = Ry (7) = 2NoB sinc(2B7) (1.104) 


4 
1.12 Representation of Narrowband Noise 
l in Terms of Envelope and Phase Components 


In Section 1.11 we considered the representation of a narrowband noise n(t) in terms of 
its in-phase and quadrature components. We may also represent the noise n(t) in terms of 
its envelope and phase components as follows: 


n(t) = r(t) cos[2af.t + p(t) (1.105) 
where 
r(t) = [nq (t) + nalt (1.106) 
and 
= tan- Zot?) 
y(t) = tan [s o| (1.107) 


The function r(t) is called the envelope of n(t), and the function (t) is called the phase of 
n(t). 

The envelope r(t) and phase y(t) are both sample functions of low-pass random 
processes. As illustrated in Figure 1.18), the time interval between two successive peaks 
of the envelope r(t) is approximately 1/B, where 2B is the bandwidth of the narrowband 
noise n(t). 

The probability distributions of r(t) and (t) may be obtained from those of m;(t) 
and no(t) as follows. Let N; and Ng denote the random variables obtained by observing 
(at some fixed time) the random processes represented by the sample functions n,(t) and 
nolt), respectively. We note that N; and No are independent Gaussian random variables 
of zero mean and variance a”, and so we may express their joint probability density func- 
tion by 





1 nt + nè 
fxing(ts no) = pp ep(- Ere ż) (1.108) 


Accordingly, the probability of the joint event that N; lies between ny and n; + dn; and 
that Ng lies between no and no + dno (i.e., the pair of random variables N; and Nog lies 
jointly inside the shaded area of Figure 1.212) is given by 


1 ne + ne 
Fung no) dn: dng = In op( 78) dn; dng (1.109) 
Define the transformation (see Figure 1.212) 
nı = r cosy (1.110) 
no =rsingd : (1.111) 


In a limiting sense, we may equate the two incremental areas shown shaded in Figures 
1.21¢@ and 1.21b and thus write 


dn, dng = r dr dy (1.112) 
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{a) (b) 


FIGURE 1.21 Illustrating the coordinate system for representation of narrowband noise: (a) in 
terms of in-phase and quadrature components, and (b) in terms of envelope and phase. 


Now, let R and ¥ denote the random variables obtained by observing (at some time ż) the 
random processes represented by the envelope r(t) and phase p(t), respectively. Then, 
substituting Equations (1.110)-(1.112) into (1.109), we find that the probability of the 
random variables R and ¥ lying jointly inside the shaded area of Figure 1.21b is equal to 


ules Oia drd 
Ino P32) v 


That is, the joint probability density function of R and ¥ is 


2 
Frovlts Y) = ee eo(-25) (1.113) 


This probability density function is independent of the angle y, which means that the 
random variables R and Y are statistically independent. We may thus express fr (r, W) 
as the product of fg(r) and fy(ip). In particular, the random variable ¥ representing phase 
is uniformly distributed inside the range 0 to 27, as shown by 


fol =d DS See (1.114) 


0, elsewhere 
This leaves the probability density function of the random variable R as 


r r? = 
Aine yep 7= (1.115) 


0, elsewhere 
where g? is the variance of the original narrowband noise n(¢). A random variable having 


the probability density function of Equation (1.115) is said to be Rayleigh distributed.* 
For convenience of graphical presentation, let 


i= (1.116) 


o 
fei) = ofrir) (1.117) 


1.13 Sine Wave Plus Narrowband Noise 69 





0.8 
0.6 
= 04 
«È 
0.2 
ames l 
[o 1 2 3 


v 


FIGURE 1.22 Normalized Rayleigh distribution. 


Then we may rewrite the Rayleigh distribution of Equation (1.115) in the normalized form 


v? 


fv) = veo(-5), uae (1.118) 
0, elsewhere 
Equation (1.118) is plotted in Figure 1.22. The peak value of the distribution fy(v) occurs 
at v=1 and is equal to 0.607. Note also that, unlike the Gaussian distribution, the Rayleigh 
distribution is zero for negative values of v. This is because the envelope r(t) can assume 
only nonnegative values. 


i 1.13 Sine Wave Plus Narrowband Noise 


Suppose next that we add the sinusoidal wave A cos(27f.t) to the narrowband noise n(t), 
where A and f, are both constants. We assume that the frequency of the sinusoidal wave 
is the same as the nominal carrier frequency of the noise. A sample function of the sinu- 
soidal wave plus noise is then expressed by 

x(t) = A cos(2f.t) + n(t) (1.119) 


Representing the narrowband noise x(t) in terms of its in-phase and quadrature compo- 
nents, we may write 
x(t) = nj(t) cos(2af,t) — molt) sin(2zf,t) (1.120) 
where 
nlt) = A + n,(t) (1.121) 
We assume that (t) is Gaussian with zero mean and variance o*. Accordingly, we may 
state the following: 


1. Both 7;(t) and no(t) are Gaussian and statistically independent. 
2. The mean of ;(t) is A and that of no(t) is zero. 
3. The variance of both 7j(t) and no(t) is o°. 


We may therefore express the joint probability density function of the random variables 
N; and Np, corresponding to #}(t) and no(t), as follows: 


1 -AP +n 
ÍNiNolnh no) = 7 op -Ai +7) (1.122) 
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Let r(t) denote the envelope of x(t) and y(t) denote its phase. From Equation (1.120), 
we thus find that 


r(t) = {HO + male)? (1.123) 
and 
w(t) = can 20] (1.124) 
ny({t) 


Following a procedure similar to that described in Section 1.12 for the derivation of the 
Rayleigh distribution, we find that the joint probability density function of the random 
variables R and ¥, corresponding to r(t) and y(t) for some fixed time ż, is given by 


2 2 
+A zarcosv) (1.125) 


fan = zl- n 


We see that in this case, however, we cannot express the joint probability density function 
frælr, W) as a product frir) fely). This is because we now have a term involving the values 
of both random variables multiplied together as r cos y. Hence, R and ¥ are dependent 
random variables for nonzero values of the amplitude A of the sinusoidal wave component. 

We are interested, in particular, in the probability density function of R. To determine 
this probability density function, we integrate Equation (1.125) over all possible values of 
y obtaining the marginal density 


2T 


frir) = è fralr, Y) dy 


m+ A2\ ("fA 
= a a(-7#) Í oxo( cos v) dy 


The integral in the right-hand side of Equation (1.126) can be identified in terms of the 
defining integral for the modified Bessel function of the first kind of zero order (see Ap- 
pendix 3); that is, 


(1.126) 


2 


I(x) = FAN exp(x cosy) dy (1.127) 


Thus, letting x = Ar/o”, we may rewrite Equation (1.126) in the compact form: 


Felt) 5 ex ae *\io(#5) (1.128) 


This relation is called the Rician distribution.’ 
As with the Rayleigh distribution, the graphical presentation of the Rician distribu- 
tion is simplified by putting 








(1.129) 


al> ain 


(1.130) 
fou) = ofrir) (1.131) 
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FIGURE 1,23 Normalized Rician distribution. 


Then we may express the Rician distribution of Equation (1.128) in the normalized form 
2 


fine vero(— > a Jilan) (1.132) 





which is plotted in Figure 1.23 for the values 0, 1, 2, 3, 5, of the parameter a. Based on 
these curves, we may make the following observations: 


1, When a is zero, the Rician distribution reduces to the Rayleigh distribution. 

2. The envelope distribution is approximately Gaussian in the vicinity of v = a when a 
is large, that is, when the sine-wave amplitude A is large compared with ø, the square 
root of the average power of the noise z(t). 


1.14 Computer Experiments: 
Flat-Fading Channel 


In this section we use computer simulations to study a multipath channel characterized by 
Rayleigh fading, examples of which arise in wireless communications and long-range radio 
transmission via the ionosphere. Fading occurs because of interference between different 
versions of the transmitted signal, which reach the receiver at correspondingly different 
times. The net result is that the received signal can vary widely in both amplitude and 
phase. Under certain conditions, the statistical time-varying nature of the received signal’s 
envelope is closely described by a Rayleigh distribution as demonstrated herein. 

Figure 1.24 presents a model of a multipath channel. It consists of a large collection 
of scatterers randomly positioned in space, whereby a single incident beam is converted 
into a correspondingly large number of scattered beams at the receiving antenna. The 
transmitted signal is set equal to A cos(27f,). It is assumed that all the scattered beams 
travel at the same mean velocity. However, they differ from each other in amplitude and 
phase by virtue of differences in path loss and path delay. Thus the kth scattered beam is 

>given by A, cos(27f,t + @,), where the amplitude A, and phase @, are random variables 
that vary slowly with time. Moreover, the @, are all independent of one another and 
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FIGURE 1.24 Model of a multipath channel. 


uniformly distributed inside the interval (0, 277]. The type of fading exhibited by the mul- 
tipath channel described herein is referred to as “flat fading” because the spectral char- 
acteristics of the transmitted signal are completely preserved at the channel output. How- 
ever, the strength of the channel output changes with time due to random fluctuations in 
the gain of the channel caused by the multipath phenomenon. 

Summing the contributions of all the scatterers, assumed to be N in number, we may 
express the random process representing the received signal as 


N 
X(t) = > A, cos(2af.t + Oy) (1.33) 
=i 
which may be rewriten in the equivalent form 
X(t) = X; cos(2af,t) — Xo sin(27f.2) (1.134) 
where the X; and Xo are respectively defined by 
N 
X; = J, A; cos @, (1.135) 
kat 
and 
N 
Xo = D Ay sin® (1.136) 
i=l 


For convenience of presentation and without loss of generality, we may assume that A, 
lies in the closed interval [—1, 1] for all k. 


Experiment 1. Gaussian Distributions 


From the central limit theorem we note that as the number of scatterers, N, approaches 
infinity, both X; and Xg should approach Gaussian random variables. To test the validity 
of this statement, the probability distributions of the in-phase component X; and quad- 
rature component Xo are computed for N = 10, 100, 1000, and 10,000. To test the 
validity of the central limit theorem, we need a measure of the goodness-of-fit that tests 
the equivalence of the measured probability distribution of the sampled data for varying 
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N to the theoretical Gaussian distribution. One way of performing such a test is to use 
central moments of a distribution (up to order 4) to define the following two parameters: 


2 
H3 
= 5 1.137 
Bi £ ( ) 
and 
B= (1.138) 
H2 


where u2, Hs, and p24 are the second, third, and fourth central moments, respectively. The 
parameters £, and £, together provide a measure of the skewness of the distribution under 
test. The closer the values £, and £; for the measured distribution are to the corresponding 
ones for the theoretical distribution, the better is the goodness-of-fit. For a Gaussian ran- 
dom variable X of mean px and variance o% we have 


H2 = ox 
#3 = 0 
H4 = 30% 
which yield 
8, =0 
and 
Bo = 3 


Table 1.1 presents the values of £, and £, computed for both the in-phase component X; 
and quadrature component Xo for varying N. Comparing these values with the corre- 
sponding ones for a Gaussian distribution, we clearly see that as the number of scatterers, 
N, increases the distributions of both X; and Xo do approach a zero-mean Gaussian 
distribution in accordance with the central limit theorem. 


i TABLE 1.1 B Values for in-phase and quadrature components 


(a) Measured Distribution 


Number of Scatterers, N 
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2.8759 2.8587 3.0075 
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In-phase component X; Bı 0.2443 | 0.0255 0.0065 0.0003 


Quadrature component Xo 




















(b) Theoretical Distribution: Gaussian 
B, = 0 


B =3 
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Experiment 2. Rayleigh Distribution 


In Equation (1.134) the random process X(t) is expressed in terms of its in-phase and 
quadrature components. Equivalently, we may express X(t) in terms of its envelope and 
phase as 


X(t) = R cos(2af,t + Y) (1.139) 
where 
R = VX? +X} (1.140) 
and 
x 
= -1| 72 
WY = tan (32) (1.141) 


Note that in the experiments considered here the in-phase component X;, quadrature 
component Xo, envelope R, and phase Y are all independent of time. 

If X; and Xo approach Gaussian random variables for increasing N, then from the 
theory presented in Section 1.12 we note that the envelope R will approach a Rayleigh 
distribution, and the phase Y will approach a uniform distribution. In Figure 1.25 we 
present the actual probability density function of r for data generated for the case of 
N = 10,000, with 100 histograms and 100 ensemble averages being computed. This figure 
also includes the theoretical curve. There is close agreement between these two curves, 
substantiating the assertion that the envelope R of the received signal approaches a Ray- 
leigh distribution. 

Figure 1.26 illustrates the effect of Rayleigh fading on the waveform of the received 
signal x(t), a sample function of X(t), for the case of a sinusoidal transmitted signal with 
unit amplitude (i.e., A = 1) and frequency f, = 1 MHz. Specifically, the transmitted signal 
and the corresponding received signal are shown in parts a and b of Figure 1.26, respec- 
tively. Comparing these two waveforms, we sce that transmission through the multipath 
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FIGURE 1.25 Probability density function of the envelope of random process X(t): comparing 
theory and experiment. 
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FIGURE 1.26 Effect of Rayleigh fading on a sinusoidal wave. (a) Input sinusoidal wave. 
(b) Waveform of the resulting signal. 


channel of Figure 1.24 has resulted in a received signal whose amplitude and phase com- 
ponents vary randomly with time, as expected. 


[| 1.15 Summary and Discussion 


Much of the material presented in this chapter has dealt with the characterization of a 
particular class of random processes known to be stationary and ergodic. The implication 
of (wide-sense) stationarity is that we may develop a partial description of a random 
process in terms of two ensemble-averaged parameters: (1) a mean that is independent of 
time, and (2) an autocorrelation function that depends only on the difference between the 
times at which two observations of the process are made.'° Ergodicity enables us to use 
time averages as “estimates” of these parameters. The time averages are computed using 
a sample function (i.e., single realization) of the random process. 

Another important parameter of a random process is the power spectral density. The 
autocorrelation function and the power spectral density constitute a Fourier-transform 
pair. The formulas that define the power spectral density in terms of the autocorrelation 
function and vice versa are known as the Einstein-Wiener-Khintchine relations. 

In Table 1.2 we present a graphical summary of the autocorrelation functions and 
power spectral densities of important random processes. All the processes described in this 
table are assumed to have zero mean and unit variance. This table should give the reader 
a feeling for (1) the interplay between the autocorrelation function and power spectral 
density of a random process, and (2) the role of linear filtering in shaping the autocorre- 
lation function or, equivalently, the power spectral density of a white noise process. 

The latter part of the chapter dealt with a noise process that is Gaussian and nar- 
rowband, which is the kind of filtered noise encountered at the front end of an idealized 
form of communication receiver. Gaussianity means that the random variable obtained by 
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‘ABLE 1.2 Graphical summary of autocorrelation functions and power spectral 


| ses fondo pr 


densities of random processes of zero mean and unit variance 


Type of Process, X(t) 
Sinusoidal process of unit 


frequency and random 
phase 


Random binary wave of unit 
symbol-duration 


RC low-pass filtered white 
noise 


Ideal low-pass filtered white 
noise 


Ideal band-pass filtered 
white noise 


RLC-filtered white noise 











Autocorrelation Function, Rx(t) 











Power Spectral Density, Sx(f) 
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observing the output of the filter at some fixed time has a Gaussian distribution. The 
narrowband nature of the noise means that it may be represented in terms of an in-phase 
and a quadrature component. These two components are both low-pass, Gaussian pro- 
cesses, each with zero mean and a variance equal to that of the original narrowband noise. 
Alternatively, a Gaussian narrowband noise may be represented in terms of a Rayleigh- 
distributed envelope and a uniformly distributed phase. Each of these representations has 
its own specific area of application, as shown in subsequent chapters of the book. 


i NOTES AND REFERENCES 


1. For a rigorous treatment of random processes, see the classic books of Doob (1953), Loéve 
(1963), and Cramér and Leadbetter (1967). 


2. There is another important class of random processes commonly encountered in practice, 
the mean and autocorrelation function of which exhibit periodicity, as in 


xlt + T) = uxt) 
Rx(ty + T, h + T) = Rx(t, t) 


for all t; and ż. A random process X(t) satisfying this pair of conditions is said to be 
cyclostationary (in the wide sense). Modeling the process X(z) as cyclostationary adds a 
new dimension, namely, period T to the partial description of the process, Examples of 
cyclostationary processes include a television signal obtained by raster-scanning a random 
video field, and a modulated process obtained by varying the amplitude, phase, or fre- 
quency of a sinusoidal carrier. For detailed discussion of cyclostationary processes, see 
Franks (1969), pp. 204-214, and the paper by Gardner and Franks (1975). 


Traditionally, Equations (1.42) and (1.43) have been referred to in the literature as the 
Wiener-Khintchine relations in recognition of pioneering work done by Norbert Wiener 
and A. I. Khintchine; for their original papers, see Wiener (1930) and Khintchine (1934). 
A discovery of a forgotten paper by Albert Einstein on time-series analysis (delivered at the 
Swiss Physical Society’s February 1914 meeting in Basel) reveals that Einstein had discussed 
the autocorrelation function and its relationship to the spectral content of a time series 
many years before Wiener and Khintchine. An English translation of Einstein’s paper is 
reproduced in the IEEE ASSP Magazine, vol. 4, October 1987. This particular issue also 
contains articles by W. A. Gardner and A. M. Yaglom, which elaborate on Einstein’s 
original work. 


ed 


4. For further details of power spectrum estimation, see Blackman and Tukey (1958), Box 
and Jenkins (1976), Marple (1987), and Kay (1988). 


5. The Gaussian distribution and associated Gaussian process are named after the great math- 
ematician C. F. Gauss. At age 18, Gauss invented the method of least squares for finding 
the best value of a sequence of measurements of some quantity. Gauss later used the method 
of least squares in fitting orbits of planets to data measurements, a procedure that was 
published in 1809 in his book entitled Theory of Motion of the Heavenly Bodies. In con- 
nection with the error of observation, he developed thé Gaussian distribution. This distri- 
bution is also known as the normal distribution. Partly for historical reasons, mathemati- 
cians commonly use the term normal, while engineers and physicists commonly use the 
term Gaussian. 


6. For a detailed treatment of electrical noise, see Van der Ziel (1970) and the collection of 
papers edited by Gupta (1977). 


An introductory treatment of shot noise is presented in Helstrom (1990). For a more de- 
tailed treatment, see the paper by Yue, Luganani, and Rice (1978). 
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10. 


| PROBLEMS 


Thermal noise was first studied experimentally by J. B. Johnson in 1928, and for this reason 
it is sometimes referred to as the Johnson noise. Johnson’s experiments were confirmed 
theoretically by Nyquist (1928). 


. The noisiness of a receiver may also be measured in terms of the so-called noise figure. The 


relationship between the noise figure and the equivalent noise temperature is developed in 
Chapter 8. 


. The Rayleigh distribution is named after the English physicist J. W. Strutt, Lord Rayleigh. 
. The Rician distribution is named in honor of Stephen O. Rice for the original contribution 


reported in a pair of papers published in 1944 and 1945, which are reproduced in Wax 
(1954). 


The statistical characterization of communication systems presented in this book is con- 
fined to the first two moments, mean and autocorrelation function (equivalently, autoco- 
variance function) of the pertinent random process. However, when a random process is 
transmitted through a nonlinear system, valuable information is contained in higher-order 
moments of the resulting output process. The parameters used to characterize higher-order 
moments in the time domain are called cumulants, and their multidimensional Fourier 
transforms are called polyspectra. For a discussion of higher-order cumulants and polys- 
pectra and their estimation, see the paper by Nikias and Raghuveer (1987). 


Stationarity and Ergodicity 
1.1 Consider a random process X(t) defined by 


X(t) = sin(27f.t) 


in which the frequency f. is a random variable uniformly distributed over the interval 
[0, W]. Show that X(t) is nonstationary. Hint: Examine specific sample functions of the 
random process X(t) for the frequency f = W/4, W/2, and W, say. 


1.2 Consider the sinusoidal process 


1.3 


X(t) = A cos(27rf.t) 
where the frequency f; is constant and the amplitude A is uniformly distributed: 
_ fi, Osas1 
fala) = if otherwise 


Determine whether or not this process is strictly stationary. 
A random process X(t) is defined by 


X(t) = A cos(2nf,t) 


where A is a Gaussian-distributed random variable of zero mean and variance gå. This 
random process is applied to an ideal integrator, producing the output 


Y(t) = Í X(t) dr 


(a) Determine the probability density function of the output Y(ż) at a particular time ty. 
(b) Determine whether or not Y(t) is stationary. 
(c) Determine whether or not Y(t) is ergodic. 


1.4 Let X and Y be statistically independent Gaussian-distributed random variables, each with 


zero mean and unit variance. Define the Gaussian process 
Z(t) = X cos(2at) + Y sin(2at) 
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(a) Determine the joint probability density function of the random variables Z(t,) and 
Z(t») obtained by observing Z(t) at times t+, and ft), respectively. 
(b) Is the process Z(t) stationary? Why? 


Correlation and Spectral Density Functions 


1.5 Prove the following two properties of the autocorrelation function Rx(7) of a random 
process X(t): 
(a) If X(t) contains a DC component equal to A, then Rx(7) will contain a constant 
component equal to A”. 
(b) If X(t) contains a sinusoidal component, then Rx(7) will also contain a sinusoidal 
component of the same frequency. 
1.6 The square wave x(t) of Figure P1.6 of constant amplitude A, period To, and delay ta, 
represents the sample function of a random process X(t). The delay is random, described 
by the probability density function 


1 1 1 
~-=Ty) = tz= = T, 
0 d 2 0 


To’ 2 
t) = 440 
Frdta) 0, otherwise 


(a) Determine the probability density function of the random variable X(t,) obtained by 
observing the random process X(t) at time f,. 

(b) Determine the mean and autocorrelation function of X(t) using ensemble-averaging. 

(c) Determine the mean and autocorrelation function of X(t) using time-averaging. 

(d) Establish whether or not X(t) is stationary. In what sense is it ergodic? 


x(t) 





slg al 


FıGURE P1.6 


1.7 A binary wave consists of a random sequence of symbols 1 and 0, similar to that described 
in Example 1.3, with one basic difference: symbol 1 is now represented by a pulse of 
amplitude A volts and symbol 0 is represented by zero volts. All other parameters are the 
same as before. Show that for this new random binary wave X(t): 


(a) The autocorrelation function is 

A A? [r| 
ELIN fo E < 
AnA ( Ty VIST 
A? 
4? 


Rx(7) = 
|r| =T 


(b) The power spectral density is 
Ae AT 
Sdf) = F EN + Jo sine*(fT) 


What is the percentage power contained in the DC component of the binary wave? 
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1.8 A random process Y(t) consists of a DC component of 3/2 volts, a periodic component 
g(t), and a random component X(t). The autocorrelation function of Y(t) is shown in 
Figure P1.8. 

(a) What is the average power of the periodic component g(t)? 
(b) What is the average power of the random component X(t)? 
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1.9 Consider a pair of stationary processes X(t) and Y(t). Show that the cross-correlations 
Rxy(7) and Ryx(7) of these processes have the following properties: 
(a) Rxy(7) = Ryx(—7) 
(b): | Rxv(7)| = 71Rx(0) + Ry(0)] 
> where Rx(7) and Ry(7) are the autocorrelation functions of X(t) and Y(t), respectively. 
1.10 Consider two linear filters connected in cascade as in Figure P1.10. Let X(t) be a stationary 
process with autocorrelation function Rx(7). The random process appearing at the first 
filter output is V(t) and that at the second filter output is Y(t). 


(a) Find the autocorrelation function of Y(t). 
(b) Find the cross-correlation function Ryy(z) of V(t) and Y(z). 


vo) 
x> A h(i) yi) 


FIGuRE P1.10 














1.11 A stationary process X(t) is applied to a linear time-invariant filter of impulse response 
h(t), producing an output Y(t). 
(a) Show that the cross-correlation function Ryx(7) of the output Y(t) and the input X(t) 
is equal to the impulse response 4(r) convolved with the autocorrelation function 
Rx(7) of the input, as shown by 


Ryx(s) = | blw)Rulr — u) du 
Show that the second cross-correlation function Rxy(7) equals 
Ryy(t) = f h(—u)Rx(t — u) du 


(b) Find the cross-spectral densities Syx(f) and Sxy(f). 
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(c) Assuming that X(t) is a white noise process with zero mean and power spectral density 
N,/2, show that 


Roxi) = 32 a(n) 


Comment on the practical significance of this result. 
1.12 The power spectral density of a random process X(t) is shown in Figure P1.12. It consists 

of a delta function at f = 0 and a triangular component. 

(a) Determine and sketch the autocorrelation function Rx(1) of X(t). 

(b) What is the DC power contained in X(t)? 

(c) What is the AC power contained in X(t)? 

(d) What sampling rates will give uncorrelated samples of X(t)? Are the samples statis- 
tically independent? 


Syif) 
a(f) 


1.0 


-fo o fo 
FIGURE P1.12 


1.13 A pair of noise processes n,(t) and m(t) are related by 
m(t) = n4(t) cos(2af.t + 6) — n,(t) sin(2arft + 0) 
where f, is a constant, and 6 is the value of a random variable ® whose probability density 
function is defined by 


1 
fa(9) = 2r’ 
0, otherwise 


0s0s2r 


The noise process ⁄4(¢) is stationary and its power spectral density is as shown in Figure 
P1.13. Find and plot the corresponding power spectral density of 2(¢). 


5y,(f) 








-W o wW 
Figure P1.13 


1.14 A random telegraph signal X(t), characterized by the autocorrelation function 
Rx(7) = exp(—2v| 7|) 
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where vis a constant, is applied to the low-pass RC filter of Figure P1.14. Determine the 


power spectral density and autocorrelation function of the random process at the filter 
output. 


R 
Input Cc t Output 


Figure P1.14 


1.15 A running integrator is defined by 


where x(t) is the input, y(t) is the output, and T is the integration period. Both x(t) and 
y(t) are sample functions of stationary processes X(t) and Y(t), respectively. Show that 
the power spectral density of the integrator output is related to that of the integrator input 
as 


Syf) = T? sinc*(fT)Sx(f) 


1.16 A zero-mean stationary process X(t) is applied to a linear filter whose impulse response 
is defined by a truncated exponential: 


ae” OstsT 
b(t) = , 
o [a otherwise 


Show that the power spectral density of the filter output ¥(ż) is defined by 


Sy(f) = 2+ 4ef (1 — 2 exp(—aT) cos(2afT) + exp(—2aT))Sx(f) 


where Sx(f) is the power spectral density of the filter input. 
1.17 The output of an oscillator is described by 
X(t) = A cos(2aft — O) 


where A is a constant, and f and © are independent random variables. The probability 
density function of @ is defined by 


1 
PA= iam "50Sa 
0, otherwise 


Find the power spectral density of X(t) in terms of the probability density function of the 
frequency f. What happens to this power spectral density when the frequency f assumes 
a constant value? 


Problems 83 


Gaussian Processes 


1.18 A stationary, Gaussian process X(t) has zero mean and power spectral density Sx(f). 
Determine the probability density function of a random variable obtained by observing 
the process X(t) at some time tą. 

1.19 A Gaussian process X(t) of zero mean and variance g% is passed through a full-wave 
rectifier, which is described by the input-output relation of Figure P1.19. Show that the 
probability density function of the random variable Y(t,), obtained by observing the ran- 
dom process Y(t) at the rectifier output at time t,, is as follows: 


2 i as 2> 0 
frealY) = 4 ¥ 1 ox exp 20%)” aS 


0, y<0 








FIGURE P1.19 


1.20 Let X(t) be a zero-mean, stationary, Gaussian process with autocorrelation function 
Rx(7). This process is applied to a square-law device, which is defined by the input-output 
relation 


Y(t) = X(t) 
where Y(t) is the output. 
(a) Show that the mean of Y(t) is Rx(0). 


(b) Show that the autocovariance function of Y(t) is 2R%(7). 

1.21 A stationary, Gaussian process X(t) with mean px and variance g% is passed through two 
linear filters with impulse responses h(t) and ba(t), yielding processes Y(t) and Z(t), as 
shown in Figure P1.21. 

(a) Determine the joint probability density function of the random variables Y(t;) and 
Z(t). 

(b) What conditions are necessary and sufficient to ensure that Y(t,) and Z(t,) are statis- 
tically independent? 
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1.22 A stationary, Gaussian process X(£) with zero mean and power spectral density Sy(f) is 
applied to a linear filter whose impulse response k(t) is shown in Figure P1.22. A sample 
Y is taken of the random process at the filter output at time T. 


(a) Determine the mean and variance of Y. 
(b) What is the probability density function of Y? 


A) 


se 


Q T 
FıGuRre P1.22 


Noise 
1.23 Consider a white Gaussian noise process of zero mean and power spectral density No/2 
that is applied to the input of the high-pass RL filter shown in Figure P1.23. 


(a) Find the autocorrelation function and power spectral density of the random process 
at the output of the filter. 


(b) What are the mean and variance of this output? 


Input L Output 
FIGURE P1.23 


1.24 A white noise w(t) of power spectral density No/2 is applied to a Butterworth low-pass 
filter of order n, whose magnitude response is defined by 


1 
MAI = EGP 


(a) Determine the noise equivalent bandwidth for this low-pass filter. (See Appendix 2 
for the definition of noise equivalent bandwidth.) 
(b) What is the limiting value of the noise equivalent bandwidth as approaches infinity? 
1.25 The shot-noise process X(t) defined by Equation (1.86) is stationary. Why? 


1.26 White Gaussian noise of zero mean and power spectral density N,/2 is applied to the 
filtering scheme shown in Figure P1.26a. The frequency responses of these two filters are 
shown in Figure P1.26b. The noise at the low-pass filter output is denoted by n(¢). 


(a) Find the power spectral density and the autocorrelation function of n(t). 
(b) Find the mean and variance of z(t). 
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(c) What is the rate at which n(t) can be sampled so that the resulting samples are essen- 
tially uncorrelated? 
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1.27 Let X(t) be a stationary process with zero mean, autocorrelation function Rx(7), and 
power spectral density Sx(f). We are required to find a linear filter with impulse response 
h(t), such that the filter output has the same statistical characteristics as X(t) when the 
input is white noise of power spectral density N,/2. 
(a) Determine the condition which the impulse response h(t) must satisfy to achieve this 
requirement. 
(b) What is the corresponding condition on the frequency response H(f) of the filter? 


Narrowband Noise 


1.28 In the noise analyzer of Figure 1.192, the low-pass filters are ideal with a bandwidth equal 
to one-half that of the narrowband noise x(t) applied to the input. Using this scheme, 
derive the following results: 

(a) Equation (1.101), defining the power spectral densities of the in-phase noise com- 
ponent 7,(t) and quadrature noise component no(t) in terms of the power spectral 
density of n(t). 

(b) Equation (1.102), defining the cross-spectral densities of 7;(¢) and no(t). 

1.29 Assume that the narrowband noise n(t) is Gaussian and its power spectral density Sn(f) 
is symmetric about the midband frequency f.. Show that the in-phase and quadrature 
components of x(t) are statistically independent. 

1.30 The power spectral density of a narrowband noise n(t) is as shown in Figure P1.30. The 
carrier frequency is 5 Hz. 

(a) Find the power spectral densities of the in-phase and quadrature components of v(t). 

(b) Find their cross-spectral densities. 


Sy(f) 
(W/Hz) 


F(Hz) 





FIGURE P1.30 
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1.31 Consider a Gaussian noise »(t) with zero mean and the power spectral density Sy(f} 
shown in Figure P1.31. 
(a) Find the probability density function of the envelope of x(t). 
(b) What are the mean and variance of this envelope? 


Sy(f) 
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Computer Experiments 


1.32 In this computer experiment we study the statistical characterization of a random process 


X(t) defined by 
X(t) = A cos(2af,t + ©) + Wit) 


where the phase @ of the sinusoidal component is a uniformly distributed random variable 
over the interval [—2, 7], and W(t) is a white Gaussian noise component of zero mean 
and power spectral density No/2. The two components of X(t) are statistically indepen- 
dent; hence the autocorrelation function of X(z) is 


2 
Rx(7) = i cos(2af.t) + > 8(7) 


This equation shows that for |r| > 0 the autocorrelation function Rx(7) has the same 

sinusoidal waveform as the signal component of X(t). 

The purpose of this computer experiment is to perform the computation of Rx(7) 
using two different methods: 

(a) Ensemble averaging. Generate M = 50 randomly picked realizations of the process 
X(t). Hence compute the product x(t + 7)x(t) for some fixed time t, where x(t) is a 
realization of X(t). Repeat the computation of x(t + 7)x(t) for the M realizations of 
X(t), and thereby compute the average of these computations over M. Repeat this 
sequence of computations for- different values of T. 

(b) Time averaging. Compute the time-averaged autocorrelation function 


T 
R,(7, T) = = (i x(t + 7)x(t) dt 


where x(t) is a particular realization of X(t), and 2T is the total observation interval. 
For this computation, use the Fourier-transform pair: 


Ral, T) = XAP)? 
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where | X7(f)|?/2T is the periodogram of the process X(t). Specifically, compute the 
Fourier transform X7(f) of the time-windowed function 
x(t), -Tsis T 
ti = 
xrl) T otherwise 


Hence compute the inverse Fourier transform of | X{f)|7/2T. 


Compare the results of your computation of Rx(7) using these two approaches. 

1.33 In this computer experiment we continue the study of the multipath channel described in 
Section 1.14. Specifically, consider the situation where the received signal includes a line- 
of-sight component, as shown by 


N 
X(t) = Ð A, cos(2nft + @,) + a cos(2mfz) 
it 
where a cos(27f,t) is the directly received component. Following the material presented 


in Section 1.14, compute the envelope of X(t) for N = 10,000, and a = 0, 1, 2, 3, 5. 
Compare your results with the Rician distribution studied in Section 1.13. 








CONTINUOUS-WAVE 
MODULATION 


In this chapter we study continuous-wave modulation, which is basic to the operation of 


analog communication systems. The chapter is divided into two related parts. In the first 
part we study the time-domain and frequency-domain descriptions of two basic families of 
continuous-wave modulation: 


> Amplitude modulation, in which the amplitude of a sinusoidal carrier is varied in 
accordance with an incoming message signal. 


® Angle modulation, in which the instantaneous frequency or phase of the sinusoidal carrier 


is varied in accordance with the message signal. 


The second part of the chapter focuses on the effects of channel noise on the performance 
of the receivers pertaining to these modulation schemes. 


Advantages and disadvantages of the different methods of continuous-wave 


modulation are highlighted in light of the material presented herein. 


i 2.1 Introduction 
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The purpose of a communication system is to transmit information-bearing signals through 
a communication channel separating the transmitter from the receiver. Information- 
bearing signals are also referred to as baseband signals. The term baseband is used to 
designate the band of frequencies representing the original signal as delivered by a source 
of information. The proper use of the communication channel requires a shift of the range 
of baseband frequencies into other frequency ranges suitable for transmission, and a cor- 
responding shift back to the original frequency range after reception. For example, a radio 
system must operate with frequencies of 30 kHz and upward, whereas the baseband signal 
usually contains frequencies in the audio frequency range, and so some form of frequency- 
band shifting must be used for the system to operate satisfactorily. A shift of the range of 
frequencies in a signal is accomplished by using modulation, which is defined as the process 
by which some characteristic of a carrier is varied in accordance with a modulating wave 
(signal). A common form of the carrier is a sinusoidal wave, in which case we speak ofa 
continuous-wave modulation’ process. The baseband signal is referred to as the modulat- 
ing wave, and the result of the modulation process is referred to as the modulated wave. 
Modulation is performed at the transmitting end of the communication system. At the 
receiving end of the system, we usually require the original baseband signal to be restored. 
This is accomplished by using a process known as demodulation, which is the reverse of. 
the modulation process. 
In basic signal-processing terms, we thus find that the transmitter of an analog com- 

munication system consists of a modulator and the receiver consists of a demodulator, as 
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Sinusoidal 
carrier wave 
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FIGURE 2.1 Components of a continuous-wave modulation system: (a) transmitter, and (b) 
receiver. 


depicted in Figure 2.1. In addition to the signal received from the transmitter, the receiver 
input includes channel noise. The degradation in receiver performance due to channel noise 
is determined by the type of modulation used. 

In this chapter we study two families of continuous-wave (CW) modulation systems, 
namely, amplitude modulation and angle modulation. In amplitude modulation, the am- 
plitude of the sinusoidal carrier wave is varied in accordance with the baseband signal. In 
angle modulation, the angle of the sinusoidal carrier wave is varied in accordance with the 
baseband signal. Figure 2.2 displays the waveforms of amplitude-modulated and angle- 
modulated signals for the case of sinusoidal modulation. Parts (a) and (b) of the figure 
show the sinusoidal carrier and modulating waves, respectively. Parts (c) and (d) show the 


(a) 





(b) 





(a) Time —> 


FIGURE 2.2 Illustrating AM and FM signals produced by a single tone. (a) Carrier wave. (b) 
Sinusoidal modulating signal. (c) Amplitude-modulated signal. (4) Frequency-modulated signal. 
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corresponding amplitude-modulated and frequency-modulated waves, respectively; fre- 
quency modulation is a form of angle modulation. This figure clearly illustrates the basic 
differences between amplitude modulation and angle modulation, which are discussed in 
what follows. 


| 2.2 Amplitude Modulation 


Consider a sinusoidal carrier wave c(t) defined by 
c(t) = A, cos(27f,t) (2.1) 


where A, is the carrier amplitude and f, is the carrier frequency. To simplify the exposition 
without affecting the results obtained and conclusions reached, we have assumed that the 
phase of the carrier wave is zero in Equation (2.1). Let m(t} denote the baseband signal 
that carries the specification of the message. The source of carrier wave c(t) is physically 
independent of the source responsible for generating m(t). Amplitude modulation (AM) is 
defined as a process in which the amplitude of the carrier wave c(t) is varied about a mean 
value, linearly with the baseband signal m(t). An amplitude-modulated (AM) wave may 
thus be described, in its most general form, as a function of time as follows: 


s(t) = A[1 + k,m(t)| cos(2af.t) (2.2) 


where k, is a constant called the amplitude sensitivity of the modulator responsible for the 
generation of the modulated signal s(t). Typically, the carrier amplitude A, and the message 
signal m(t) are measured in volts, in which case k, is measured in volt™?. 

Figure 2.3¢ shows a baseband signal m(t), and Figures 2.3b and 2.3c show the cor- 
responding AM wave s(t) for two values of amplitude sensitivity k, and a carrier amplitude 
A, = 1 volt. We observe that the envelope of s(t) has essentially the same shape as the 
baseband signal m(t) provided that two requirements are satisfied: 


1, The amplitude of k(t) is always less than unity, that is, 
|km(t)|<1 for allt (2.3) 


This condition is illustrated in Figure 2.30; it ensures that the function 1 + k,#m(t) 
is always positive, and since an envelope is a positive function, we may express the 
envelope of the AM wave s(t) of Equation (2.2) as A.[1 + k,7m(t)]. When the am- 
plitude sensitivity k, of the modulator is large enough to make |k,m/(t)| > 1 for any 
t, the carrier wave becomes overmodulated, resulting in carrier phase reversals when- 
ever the factor 1 + k,1m{(t) crosses zero. The modulated wave then exhibits envelope 
distortion, as in Figure 2.3c. It is therefore apparent.that by avoiding overmodula- 
tion, a one-to-one relationship is maintained between the envelope of the AM wave 
and the modulating wave for all values of time—a useful feature, as we shall see later 
on. The absolute maximum value of k,7n(t) multiplied by 100 is referred to as the 
percentage modulation. 


The carrier frequency f, is much greater than the highest frequency component W of 
the message signal m(t), that is 


b&b 


f.>> W (2.4) 


We call W the message bandwidth. If the condition of Equation (2.4) is not satisfied, 
an envelope cannot be visualized (and therefore detected) satisfactorily. 
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FIGURE 2.3 Illustrating the amplitude modulation process. (a) Baseband signal m(t). (b) AM 
wave for | k,m(t) | < 1 for all t. (c) AM wave for | kam(t)| > L for some t. 


From Equation (2.2), we find that the Fourier transform of the AM wave s(t) is given 
by 


sip) = Se (oi — fy + air + f+ EM V+ MET + (28) 


Suppose that the baseband signal m(t) is band-limited to the interval -W = f = W, as in 
Figure 2.4a. The shape of the spectrum shown in this figure is intended for the purpose of 
illustration only. We find from Equation (2.5) that the spectrum S(f) of the AM wave is 
as shown in Figure 2.46 for the case when f, > W. This spectrum consists of two delta 
functions weighted by the factor A,/2 and occurring at +f., and two versions of the 
baseband spectrum translated in frequency by +f, and scaled in amplitude by &,A,/2. 
From the spectrum of Figure 2.46, we note the following: 


1. As a result of the modulation process, the spectrum of the message signal m(t) for 
negative frequencies extending from — W to 0 becomes completely visible for positive 
(i.e., measurable) frequencies, provided that the carrier frequency satisfies the con- 
dition f, > W; herein lies the importance of the idea of “negative” frequencies. 

2. For positive frequencies, the portion of the spectrum of an AM wave lying above the 
carrier frequency f, is referred to as the upper sideband, whereas the symmetric 
portion below f, is referred to as the lower sideband. For negative frequencies, the 
upper sideband is represented by the portion of the spectrum below —f, and the 
lower sideband by the portion above —f,. The condition f, > W ensures that 
the sidebands do not overlap. 
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FIGURE 2.4 (a) Spectrum of baseband signal. (b) Spectrum of AM wave. 


For positive frequencies, the highest frequency component of the AM wave equals 
f. + W, and the lowest frequency component equals f- — W. The difference between 
these two frequencies defines the transmission bandwidth Br for an AM wave, which 
is exactly twice the message bandwidth W, that is, 


By =2W . (2.6) 


@ VIRTUES AND LIMITATIONS OF AMPLITUDE MODULATION 


Amplitude modulation is the oldest method of performing modulation. Its greatest virtue 
is the simplicity of implementation: 


> In the transmitter, amplitude modulation is accomplished using a nonlinear device. 


For example, in the switching modulator discussed in Problem 2.3, the combined 
sum of the message signal and carrier wave is applied to a diode, with the carrier 
amplitude being large enough to swing across the characteristic curve of the diode. 
Fourier analysis of the voltage developed across a resistive load reveals the generation 
of an AM component, which may be extracted by means of a band-pass filter. 

In the receiver, amplitude demodulation is also accomplished using a nonlinear de- 
vice. For example, we may use a simple and yet highly effective circuit known as the 
envelope detector, which is discussed in Problem 2.5. The circuit consists of a diode 
connected in series with the parallel combination of a capacitor and load resistor. 
Some version of this circuit is found in most commercial AM radio receivers. Pro- 
vided that the carrier frequency is high enough and the percentage modulation is less 
than 100 percent, the demodulator output developed across the load resistor is nearly 
the same as the envelope of the incoming AM wave, hence the name “envelope 
detector.” 


Recall, however, that transmitted power and channel bandwidth are our two primary 


communication resources, and they should be used efficiently. In this context, we find that 
the standard form of amplitude modulation defined in Equation (2.2) suffers from two 
major limitations: 


1. 


Amplitude modulation is wasteful of power. The carrier wave c(t) is completely 
independent of the information-bearing signal m(t). The transmission of the carrier 
wave therefore represents a waste of power, which means that in amplitude modu- 
lation only a fraction of the total transmitted power is actually affected by m(2). 
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2. Amplitude modulation is wasteful of bandwidth. The upper and lower sidebands of 
an AM wave are uniquely related to each other by virtue of their symmetry about 
the carrier frequency; hence, given the magnitude and phase spectra of either side- 
band, we can uniquely determine the other. This means that insofar as the transmis- 
sion of information is concerned, only one sideband is necessary, and the commu- 
nication channel therefore needs to provide only the same bandwidth as the baseband 
signal. In light of this observation, amplitude modulation is wasteful of bandwidth 
as it requires a transmission bandwidth equal to twice the message bandwidth. 


To overcome these limitations, we must make certain modifications: suppress the 
carrier and modify the sidebands of the AM wave. These modifications naturally result in 
increased system complexity. In effect, we trade system complexity for improved use of 
communication resources. The basis of this trade-off is linear modulation, which is dis- 
cussed in the next section. In a strict sense, full amplitude modulation does not qualify as 
linear modulation because of the presence of the carrier wave. 


i 2.3 Linear Modulation Schemes 


In its most general form, linear modulation is defined by 
s(t) = s;(t) cos(2af,t) — solt) sin(27f,t) (2.7) 


where sz(t) is the in-phase component of the modulated wave s(t), and so(t) is its quad- 
rature component. Equation (2.7) is recognized as the canonical representation of a nar- 
rowband signal, which is discussed in detail in Appendix 2. In linear modulation, both 
$,(t) and so(t) are low-pass signals that are linearly related to the message signal m(t}. 

Indeed, depending on how these two components of s(z) are defined, we may identify 
three types of linear modulation involving a single message signal: 


1. Double sideband-suppressed carrier (DSB-SC) modulation, where only the upper and 
lower sidebands are transmitted. 

2. Single sideband (SSB) modulation, where only one sideband (the lower sideband or 
the upper sideband) is transmitted. 

3. Vestigial sideband (VSB) modulation, where only a vestige (i.e., trace) òf. one of the 
sidebands and a correspondingly modified version of the other sideband are 
transmitted. 


Table 2.1 presents a summary of the definitions of these three special forms of linear 
modulation. There are two important points to note from Table 2.1: 


1. The in-phase component sz(ż) is solely dependent on the message signal m(t). 
2. The quadrature component so(t) is a filtered version of m(t). The spectral modifi- 
cation of the modulated wave s(t) is solely due to so(2). 


To be more specific, the role of the quadrature component (if present) is merely to interfere 
with the in-phase component, so as to reduce or eliminate power in one of the sidebands 
of the modulated signal s(t), depending on how the quadrature component is defined. 
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| TABLE 2.1 Different forms of linear modulation 














In-Phase Quadrature 
Component Component 
Type of Modulation s(t) Sa(t) Comments 
DSB-SC m(t) 0 m(t) = message signal 
SSB:* 
(a) Upper sideband imit) talt) *A(t) = Hilbert transform of m(t) 
transmitted ` 
(b) Lower sideband m(t) —49n(t) 
transmitted 
VSB: 
(a) Vestige of lower sideband im(t) dm’ (t) m’(t) = output of the filter of 
transmitted frequency response Ho(f) 
(b) Vestige of upper zm(t) —}m'(t) due to m(t). 
sideband transmitted For the definition of Ho(f), 


see Eq. (2.16) 


"For the mathematical description of single sideband modulation, see Problem 2.16. 


e DOUBLE SIDEBAND-SUPPRESSED CARRIER (DSB-SC) MODULATION 


This form of linear modulation is generated by using a product modulator that simply 
multiplies the message signal m(t) by the carrier wave A, cos(27f,t), as illustrated in Figure 
2.5a. Specifically, we write 


s(t) = An(t) cos(27f,t) (2.8) 


mli) 






DSB-SC 
modulated wave 
s(t) = Aml) cos (27f,.t) 
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modutator 
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A, COS (2r fot) 


(a) (b) 


s(t) 


Phase reversals 








{ce} 


FIGURE 2.5 (a) Block diagram of product modulator. (b) Baseband signal. (c) DSB-SC modu- 
lated wave. 
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FIGURE 2.6 (a) Spectrum of baseband signal. (b) Spectrum of DSB-SC modulated wave. 


Figure 2.5c shows the modulated signal s(t) for the arbitrary message waveform of Figure 
2.5b. The modulated signal s(t) undergoes a phase reversal whenever the message signal 
m(t) crosses zero. Consequently, the envelope of a DSB-SC modulated signal is different 
from the message signal; this is unlike the case of an AM wave that has a percentage 
modulation less than 100 percent. 
From Equation (2.8), the Fourier transform of s(t) is obtained as 

Sif) = 5 AAMUF — f) + MIF + fl (2.9) 
For the case when the baseband signal m(t) is limited to the interval -W = f = W, as in 
Figure 2.6@, we thus find that the spectrum S(f) of the DSB-SC wave s(t) is as illustrated 
in Figure 2.6. Except for a change in scale factor, the modulation process simply translates 
the spectrum of the baseband signal by +f.. Of course, the transmission bandwidth rè- 
quired by DSB-SC modulation is the same as that for amplitude modulation, namely, 2 W. 


ss COHERENT DETECTION 


The baseband signal m(t) can be uniquely recovered from a DSB-SC wave s(t) by first 
multiplying s(t) with a locally generated sinusoidal wave and then low-pass filtering the 
product, as in Figure 2.7. It is assumed that the local oscillator signal is exactly coherent 
or synchronized, in both frequency and phase, with the carrier wave c(t) used in the prod- 
uct modulator to generate s(t). This method of demodulation is known as coherent detec- 
tion or synchronous demodulation. 

It is instructive to derive coherent detection as a special case of the more general 
demodulation process using a local oscillator signal of the same frequency but arbitrary 
phase difference @, measured with respect to the carrier wave c(t). Thus, denoting the local 
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Figure 2.7 Coherent detector for demodulating DSB-SC modulated wave. 
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FIGURE 2.8 _ Illustrating the spectrum of a product modulator output with a DSB-SC modulated 
wave as input. 








oscillator signal by A} cos(27f.t + $), and using Equation (2.8) for the DSB-SC wave s(t), 
we find that the product modulator output in Figure 2.7 is 


u(t) = AL cos(2af,t + $)s(t) 
= A.A! cos(2af-t) cos(2af.t + y(t) (2.10) 


= ZAA cos(4rf,t + ġ)m(t) + 5 AA! cos ġ m(t) 


The first term in Equation (2.10) represents a DSB-SC modulated signal with a carrier 
frequency 2f., whereas the second term is proportional to the baseband signal m(t). This 
is further illustrated by the spectrum V(f) shown in Figure. 2.8, where it is assumed that 
the baseband signal m(t) is limited to the interval -W = f = W. It is therefore apparent 
that the first term in Equation (2.10) is removed by the low-pass filter in Figure 2.7, 
provided that the cut-off frequency of this filter is greater than W but less than 2f; — W. 
This requirement is satisfied by choosing f. > W. At the filter output we then obtain a 
signal given by 


v(t) = ZAA cos ġ m(t) (2.11) 


The demodulated signal v,(t) is therefore proportional to m(t) when the phase error ọ 
is a constant. The amplitude of this demodulated signal is maximum when ¢ = 0, and 
it is minimum (zero) when @ = +2/2. The zero demodulated signal, which occurs for 
= +7/2, represents the quadrature null effect of the coherent detector. Thus the phase 
error ¢ in the local oscillator causes the detector output to be attenuated by a factor equal 
to cos @. As long as the phase error ¢ is constant, the detector provides an undistorted 
version of the original baseband signal z(t). In practice, however, we usually find that the 
phase error ¢ varies randomly with time, due to random variations in the communication 
channel, The result is that at the detector output, the multiplying factor cos ¢ also varies 
randomly with time, which is obviously undesirable. Therefore, provision must be made 
in the system to maintain the local oscillator in the receiver in perfect synchronism, in both 
frequency and phase, with the carrier wave used to generate the DSB-SC modulated signal 
in the transmitter. The resulting system complexity is the price that must be paid for 
suppressing the carrier wave to save transmitter power. 


a CosTas RECEIVER 


One method of obtaining a practical synchronous receiver system, suitable for demodu- 
lating DSB-SC waves, is to use the Costas receiver? shown in Figure 2.9. This receiver 
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FIGURE 2.9 Costas receiver. 


consists of two coherent detectors supplied with the same input signal, namely, the incom- 
ing DSB-SC wave A, cos(2af,¢)m(t), but with individual local oscillator signals that are 
in phase quadrature with respect to each other. The frequency of the local oscillator is 
adjusted to be the same as the carrier frequency f., which is assumed known a priori. The 
detector in the upper path is referred to as the in-phase coherent detector or I-channel, 
and that in the lower path is referred to as the quadrature-phase coherent detector or 
Q-channel. These two detectors are coupled together to form a negative feedback system 
designed in such a way as to maintain the local oscillator synchronous with the carrier 
wave. ` 
To understand the operation of this receiver, suppose that the local oscillator signal 
is of the same phase as the carrier wave A.cos(27f,t) used to generate the incoming 
DSB-SC wave. Under these conditions, we find that the I-channel output contains the 
desired demodulated signal m(t), whereas the Q-channel output is zero due to the quad- 
rature null effect of the Q-channel. Suppose next that the local oscillator phase drifts from 
its proper value by a small angle ġ radians. The J-channel output will remain essentially 
unchanged, but there will now be some signal appearing at the Q-channel output, which 
is proportional to sin @ = ¢ for small ¢. This Q-channel output will have the same polarity 
as the I-channel output for one direction of local oscillator phase drift and opposite po- 
larity for the opposite direction of local oscillator phase drift. Thus, by combining the 
I- and Q-channel outputs in a phase discriminator (which consists of a multiplier followed 
by a low-pass filter), as shown in Figure 2.9, a DC control signal is obtained that auto- 
matically corrects for local phase errors in the voltage-controlled oscillator. 

It is apparent that phase control in the Costas receiver ceases with modulation and 
that phase-lock has to be reestablished with the reappearance of modulation. This is not 
a serious problem when receiving voice transmission, because the lock-up process normally 
occurs so rapidly that no distortion is perceptible. 


QUADRATURE-CARRIER MULTIPLEXING 


The quadrature null effect of the coherent detector may also be put to good use in the 
construction of the so-called quadrature-carrier multiplexing or quadrature-amplitude 
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FIGURE 2.10 Quadrature-carrier multiplexing system. (a) Transmitter. (b) Receiver. 


modulation (QAM). This scheme enables two DSB-SC modulated waves (resulting from 
the application of two physically independent message signals) to occupy the same channel 
bandwidth, and yet it allows for the separation of the two message signals at the receiver 
output. It is therefore a bandwidth-conservation scheme. 

A block diagram of the quadrature-carrier multiplexing system is shown in Figure 
2.10. The transmitter part of the system, shown in Figure 2.10a, involves the use of two 
separate product modulators that are supplied with two carrier waves of the same fre- 
quency but differing in phase by —90 degrees. The transmitted signal s(t) consists of the 
sum of these two product modulator outputs, as shown by 


s(t) = Aam (t) cos(2mf.t) + Asm(t) sin(27f.t) (2.12) 


where m(t) and m(t) denote the two different message signals applied to the product 
modulators. Thus s(t) occupies a channel bandwidth of 2W centered at the carrier fre- 
quency f., where W is the message bandwidth of m,(t) or m(t). According to Equation 
(2.12), we may view A,m(t) as the in-phase component of the multiplexed band-pass 
signal s(t) and —Agm,(t) as its quadrature component. 

The receiver part of the system is shown in Figure 2.10. The multiplexed signal s(t) 
is applied simultaneously to two separate coherent detectors that are supplied with two 
local carriers of the same frequency but differing in phase by —90 degrees. The output of 
the top detector is A.7,(t), whereas the output of the bottom detector is A.7(t). For the 
system to operate satisfactorily, it is important to maintain the correct phase and frequency 
relationships between the local oscillators used in the transmitter and receiver parts of the 
system. 

To maintain this synchronization, we may send a pilot signal outside the passband 
of the modulated signal. In this method, the pilot signal typically consists of a low-power 
sinusoidal tone whose frequency and phase are related to the carrier. wave c(t); at the 
receiver, the pilot signal is extracted by means of a suitably tuned circuit and then trans- 
lated to the correct frequency for use in the coherent detector. 


æ SINGLE-SIDEBAND MODULATION 


In single-sideband modulation, only the upper or lower sideband is transmitted. We may 
generate such a modulated wave by using the frequency-discrimination method that con- 
sists of two stages: 
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> The first stage is a product modulator, which generates a DSB-SC modulated wave. 


œ The second stage is a band-pass filter, which is designed to pass one of the sidebands 
of this modulated wave and suppress the other. 


From a practical viewpoint the most severe requirement of SSB generation using the fre- 
quency discrimination method arises from the unwanted sideband. The nearest frequency 
component of the unwanted sideband is séparated from the desired sideband by twice the 
lowest frequency component of the message (modulating) signal. The implication here is 
that for the generation of an SSB modulated signal to be possible, the message spectrum 
must have an energy gap centered at the origin, as illustrated in Figure 2.11a. This require- 
ment is naturally satisfied by voice signals, whose energy gap is about 600 Hz wide (i.e., 
it extends from —300 to +300 Hz). Thus, assuming that the upper sideband is retained, 
the spectrum of the SSB modulated signal is as shown in Figure 2.116. 

In designing the band-pass filter used in the frequency-discriminator for generating 
a SSB-modulated wave, we must meet the three basic requirements: 


> The desired sideband lies inside the passband of the filter. 
e The unwanted sideband lies inside the stopband of the filter. 


> The filter’s transition band, which separates the passband from the stopband, is twice 
the lowest frequency component of the message signal. 


This kind of frequency discrimination usually requires the use of highly selective filters, 
which can only be realized in practice by means of crystal resonators. 

To demodulate a SSB modulated signal s(t), we may use a coherent detector, which 
multiplies s(t) by a locally generated carrier and then low-pass filters the product. This 
method of demodulation assumes perfect synchronism between the oscillator in the co- 
herent detector and the oscillator used to supply the carrier wave in the transmitter. This 
requirement is usually met in one of two ways: 


> A low-power pilot carrier is transmitted in addition to the selected sideband. 
> A highly stable oscillator, tuned to the same frequency as the carrier frequency, is 
used in the receiver. 


In the latter method, it is inevitable that there would be some phase error @ in the local 
oscillator output with respect to the carrier wave used to generate the incoming SSB mod- 
ulated wave. The effect of this phase error is to introduce a phase distortion in the de- 
modulated signal, where each frequency component of the original message signal under- 
goes a constant phase shift ¢. This phase distortion is tolerable in voice communications, 
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FIGURE 2.11 (a) Spectrum of a message signal m(t) with an energy gap of width 2f, centered 
on the origin. (b) Spectrum of corresponding SSB signal containing the upper sideband. 
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because the human ear is relatively insensitive to phase distortion. In particular, the pres- 
ence of phase distortion gives rise to a Donald Duck voice effect. In the transmission of 
music and video signals, on the other hand, the presence of this form of waveform distor- 
tion is utterly unacceptable. 


VESTIGIAL SIDEBAND MODULATION 


In vestigial sideband (VSB) modulation, one of the sidebands is partially suppressed and 
a vestige of the other sideband is transmitted to compensate for that suppression. A popular 
method for generating a VSB-modulated wave is to use the frequency discrimination 
method. First, we generate a DSB-SC modulated wave and then pass it through a band- 
pass filter, as shown in Figure 2.12; it is the special design of the band-pass filter that 
distinguishes VSB modulation from SSB modulation. Assuming that a vestige of the lower 
sideband is transmitted, the frequency response H(f) of the band-pass filter takes the form 
shown in Figure 2.13. To simplify matters, only the response for positive frequencies is 
shown here. This frequency response is normalized, so that at the carrier frequency f, we 
have | H(f.)| = 1/2. The important feature to note from Figure 2.13 is that the cutoff 
portion of the frequency response around the carrier frequency fe exhibits odd symmetry. 
That is, inside the transition interval f. — f, = | f | = f; + f, the following two conditions 
are satisfied: 


1. The sum of the values of the magnitude response | H(f)| at any two frequencies 
equally displaced above and below f, is unity. 
2. The phase response arg(H(f)) is linear. That is, H(f) satisfies the condition 
Hif - f) + HF + f)=1 for-Wsfsw (2.13) 
Note also that outside the frequency band of interest (i.e., | f | > f- + W), the frequency 


response H(f) may have an arbitrary specification. Accordingly, the transmission band- 
width of VSB modulation is 


Br=W+ f, (2.14) 
where W is the message bandwidth, and f, is the width of the vestigial sideband. 
According to Table 2.1, the VSB modulated wave is described in the time domain as 


s(t) = 1 Aant) cos{2rf.t) + ; Agm'(t) sin(2-7f,t) (2.15) 


where the plus sign corresponds to the transmission of a vestige of the upper sideband, 
and the minus sign corresponds to the transmission of a vestige of the lower sideband. The 
signal m’(t) in the quadrature component of s(t) is obtained by passing the message signal 
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FIGURE 2.12 Filtering scheme for the generation of VSB modulated wave. 
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FIGURE 2.13 Magnitude response of VSB filter; only the positive-frequency portion is shown. 


m(t) through a filter whose frequency response Ho(f) satisfies the following requirement 
(see Problem 2.20): 


Holf) = AHF - f) = Hf + fl for-W=f=<W (2.16) 


Figure 2.14 displays a plot of the frequency response Ho(f), scaled by 1/;. The role of the 
quadrature component determined by Ho(f) is to interfere with the in-phase component 
in Equation (2.15) so as to partially reduce power in one of the sidebands of the modulated 
wave s(t) and retain simply a vestige of the other sideband, as desired. 

It is of interest to note that SSB modulation may be viewed as a special case of 
VSB modulation. Specifically, when the vestigial sideband is reduced to zero (i.e., we set 
f. = 0), the modulated wave s(t) of Equation (2.15) takes the limiting form of a single- 
sideband modulated wave. 


m ‘TELEVISION SIGNALS 


A discussion of yestigial sideband modulation would be incomplete without a mention of 
its role in commercial television (TV) broadcasting. The exact details of the modulation 
format used to transmit the video signal characterizing a TV system are influenced by two 
factors: 


1. The video signal exhibits a large bandwidth and significant low-frequency content, 
which suggest the use of vestigial sideband modulation. 

2. The circuitry used for demodulation in the receiver should be simple and therefore 
inexpensive; this suggests the use of envelope detection, which requires the addition 
of a carrier to the VSB-modulated wave. 








FIGURE 2.14 Frequency response of a filter for producing the quadrature component of the 
VSB modulated wave. 
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With regard to point 1, however, it should be stressed that although there is indeed 
a basic desire to conserve bandwidth, in commercial TV broadcasting the transmitted 
signal is not quite VSB modulated. The reason is that at the transmitter the power levels 
are high, with the result that it would be expensive to rigidly control the filtering of side- 
bands. Instead, a VSB filter is inserted in each receiver, where the power levels are low, 
The overall performance is the same as conventional vestigial-sideband modulation, except 
for some wasted power and bandwidth. These remarks are illustrated in Figure 2.15. In 
particular, Figure 2.15a shows the idealized spectrum of a transmitted TV signal. The 
upper sideband, 25 percent of the lower sideband, and the picture carrier are transmitted, 
The frequency response of the VSB filter used to do the required spectrum shaping in the 
receiver is shown in Figure 2.150. 

The channel bandwidth used for TV broadcasting in North America is 6 MHz, as 
indicated in Figure 2.156. This channel bandwidth not only accommodates the bandwidth 
requirement of the VSB modulated video signal but also provides for the accompanying 
sound signal that modulates a carrier of its own. The values presented on the frequency 
axis in Figures 2.154 and 2.156 pertain to a specific TV channel. According to this figure, 
the picture carrier frequency is at 55.25 MHz, and the sound carrier frequency is at 59.75 
MHz. Note, however, that the information content of the TV signal lies in a baseband 
spectrum extending from 1.25 MHz below the picture carrier to 4.5 MHz above it. 

With regard to point 2, the use of envelope detection (applied to a VSB modulated 
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-FIGURE 2.15 (a) Idealized magnitude spectrum of a transmitted TV signal. (b) Magnitude re- 
sponse of VSB shaping filter in the receiver. 
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wave plus carrier) produces waveform distortion in the video signal recovered at the de- 
tector output. The distortion is produced by the quadrature component of the VSB mod- 
ulated wave; this issue is discussed next. 

The use of the time-domain description given in Equation (2.15) enables the deter- 
mination of the waveform distortion caused by the envelope detector. Specifically, adding 
the carrier component A, cos(27f,t) to the VSB-modulated wave of Equation (2.15), the 
latter being scaled by a factor k,, modifies the modulated signal applied to the envelope 
detector input as f 


s(t) = alı + 1 komt cos(2af.t) + : k,Agn'(t) sin(27f.t) (2.17) 


where the constant k, determines the percentage modulation. The envelope detector out- 
put, denoted by a(t), is therefore 


1 2 1 25 1/2 
a(t) = adl: + 3 kam) + G tmin | } 


2) 1/2 2.18 
1 x kam'(t) ( ) 
= ali + z kam 1+ 


1+ 2 kamit) 


Equation (2.18) indicates that the distortion is contributed by m’ (t), which is responsible 
for the quadrature component of the incoming VSB-modulated signal. This distortion can 
be reduced by using two methods: 


> Reducing the percentage modulation to reduce the amplitude sensitivity k,. 
> Increasing the width of the vestigial sideband to reduce m’ (t). 


Both methods are in fact used in practice. In commercial TV broadcasting, the width of 
the vestigial sideband (which is about 0.75 MHz, or one-sixth of a full sideband) is deter- 
mined to keep the distortion due to m'(¢) within tolerable limits when the percentage 
modulation is nearly 100. 


| 2.4 Frequency Translation 


The basic operation involved in single-sideband modulation is in fact’a form of frequency 
translation, which is why single-sideband modulation is sometimes referred to as frequency 
changing, mixing, or heterodyning. This operation is clearly illustrated in the spectrum of 
the signal shown in Figure 2.11b compared to that of the original message signal in Figure 
2.112. Specifically, we see that a message spectrum occupying the band from f, to f, for 
positive frequencies in Figure 2.114 is shifted upward by an amount equal to the carrier 
frequency f, in Figure 2.11b, and the message spectrum for negative frequencies is trans- 
lated downward in a symmetric fashion. 

The idea of frequency translation described herein may be generalized as follows. 
Suppose that we have a modulated wave s,(t) whose spectrum is centered on a carrier 
frequency fı, and the requirement is to translate it upward in frequency such that its carrier 
frequency is changed from f, to a new value fọ. This requirement may be accomplished 
using the mixer shown in Figure 2.16. Specifically, the mixer is a device that consists of a 
product modulator followed by a band-pass filter. 
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FIGURE 2.16 Block diagram of mixer. 


To explain the action of the mixer, consider the situation depicted in Figure 2.17, 
where, for the purpose of illustration, it is assumed that the mixer input s,(¢) is an AM 
signal with carrier frequency fı and bandwidth 2W. Part (a) of Figure 2.17 displays the 
AM spectrum S,(f) assuming that fı > W. Part (b) of the figure displays the spectrum 
S'(f) of the resulting signal s’(t) at the product modulator output. 

The signal s’(t) may be viewed as the sum of two modulated components: one com- 
ponent represented by the shaded spectrum in Figure 2.17b, and the other component 
represented by the unshaded spectrum in this figure. Depending on whether the incoming 
carrier frequency fı is translated upward or downward, we may identify two different 
situations, as described here: 


Up conversion. In this case the translated carrier frequency f is greater than the 
incoming carrier frequency fı, and the required local oscillator frequency fi is there- 


fore defined by 
h=fth 
or 
fh=h-h 
sf) 


Sie: | 


{a) 


sf) 





ith o r = ; 
EE A 


FIGURE 2.17 (a) Spectrum of modulated signal sı(t) at the mixer input. (b) Spectrum of the 
corresponding signal s'(t) at the output of the product modulator in the mixer. 
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The unshaded part of the spectrum in Figure 2.17b defines the wanted modulated 
signal s(t), and the shaded part of this spectrum defines the image signal associated 
with s,(t). For obvious reasons, the mixer in this case is referred to as a frequency- 
up converter. 

Down conversion. In this second case the translated carrier frequency f, is smaller 
than the incoming carrier frequency fı, and the required oscillator frequency f; is 


therefore defined by 
h=f-h 


or 


f=h- he 


The picture we have this time is the reverse of that pertaining to up conversion. In 
particular, the shaded part of the spectrum in Figure 2.17 defines the wanted mod- 
ulated signal s,(t), and the unshaded part of this spectrum defines the associated 
image signal. The mixer is now referred to as a frequency-down converter. Note that 
in this case the translated carrier frequency f, has to be larger than W (i.e., one half 
of the bandwidth of the modulated signal) to avoid sideband overlap. 


The purpose of the band-pass filter in the mixer of Figure 2.16 is to pass the wanted 
modulated signal s(t) and eliminate the associated image signal. This objective is achieved 
by aligning the midband frequency of the filter with the translated carrier frequency fy and 
assigning it a bandwidth equal to that of the incoming modulated signal s,(t). 

It is important to note that mixing is a linear operation. Accordingly, the relation of 
the sidebands of the incoming modulated wave to the carrier is completely preserved at 
the mixer output. 


| 2.5 Frequency-Division Multiplexing 


Another important signal processing operation is multiplexing, whereby a number of in- 
dependent signals can be combined into a composite signal suitable for transmission over 
a common channel. Voice frequencies transmitted over telephone systems, for example, 
range from 300 to 3100 Hz. To transmit a number of these signals over the same channel, 
the signals must be kept apart so that they do not interfere with each other, and thus they 
can be separated at the receiving end. This is accomplished by separating the signals either 
in frequency or in time. The technique of separating the signals in frequency is referred to 
as frequency-division multiplexing (FDM), whereas the technique of separating the signals 
in time is called time-division multiplexing (TDM). In this section, we discuss FDM sys- 
tems, and TDM systems are discussed in Chapter 3. 

A block diagram of an FDM system is shown in Figure 2.18. The incoming message 
signals are assumed to be of the low-pass type, but their spectra do not necessarily have 
nonzero values all the way down to zero frequency. Following each signal input, we have 
shown a low-pass filter, which is designed to remove high-frequency components that do 
not contribute significantly to signal representation but are capable of disturbing other 
message signals that share the common channel. These low-pass filters may be omitted 
only if the input signals are sufficiently band limited initially. The filtered signals are applied 






























































106 CHAPTER 2 & CONTINUOUS-WAVE MODULATION 
Message Low-pass. Band-pass Band-pass Low-pass Message 
inputs filters Modulators filters filters Demodulators filters outputs 
l> LP — MOD > BP BP —— DEM — LP ee 1 
2—> LP BP m> IP p> 2 
Common 
channel 
NH= LP BP > LP >N 





















































Carrier 
supply 


Transmitter Receiver 


FIGURE 2.18 Block diagram of FDM system. 
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to modulators that shift the frequency ranges of the signals so as to occupy mutually 
exclusive frequency intervals. The necessary carrier frequencies needed to perform these 
frequency translations are obtained from a carrier supply. For the modulation, we may 
use any one of the methods described in previous sections of this chapter. However, the 
most widely used method of modulation in frequency-division multiplexing is single side- 
band modulation, which, in the case of voice signals, requires a bandwidth that is ap- 
proximately equal to that of the original voice signal. In practice, each voice input is usually 
assigned a bandwidth of 4 kHz. The band-pass filters following the modulators are used 
to restrict the band of each modulated wave to its prescribed range. The resulting band- 
pass filter outputs are next combined in parallel to form the input to the common channel. 
At the receiving terminal, a bank of band-pass filters, with their inputs connected in par- 
allel, is used to separate the message signals on a frequency-occupancy basis. Finally, the 
original message signals are recovered by individual demodulators. Note that the FDM 
system shown in Figure 2.18 operates in only one direction. To provide for two-way 
transmission, as in telephony, for example, we have to completely duplicate the multi- 
plexing facilities, with the components connected in reverse order and with the signal 
waves proceeding from right to left. 


> EXAMPLE 2.1 


The practical implementation of an FDM system usually involves many steps of modulation 
and demodulation, as illustrated in Figure 2.19. The first multiplexing step combines 12 voice 
inputs into a basic group, which is formed by having the mth input modulate a carrier at 
frequency f, = 60 + 4n kHz, where 7 = 1, 2,..., 12. The lower sidebands are then selected 
by band-pass filtering and combined to form a group of 12 lower sidebands (one for each 
voice input). Thus the basic group occupies the frequency band 60 to 108 kHz. The next step 
in the FDM hierarchy involves the combination of five basic groups into a supergroup. This 
is accomplished by using the nth group to modulate a carrier of frequency f, = 372 + 487 
kHz, where n = 1,2,..., 5. Here again the lower sidebands are selected by filtering and then. 
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FIGURE 2.19 Illustrating the modulation Steps in an FDM system. 


combined to form a Supergroup occupying the band 312 to $52 kHz. Thus a supergroup is 
designed to accommodate 60 independent voice inputs. The reason for forming the supergroup 
in this manner is that economical filters of the required characteristics are available only over 
a limited frequency range. In a similar manner, supergroups are combined into mastergroups, 


and mastergroups are combined into very large groups. G 
-| 2.6 Angle Modulation 
T- Ee iat 


In the previous sections of this chapter, we investigated the effect of slowly varying the 
amplitude of a sinusoidal carrier wave in accordance with the baseband (information- 
carrying) signal. There is another way of modulating a sinusoidal carrier wave, namely, 
angle modulation in which the angle of the carrier wave is varied according to the baseband 
signal. In this method of modulation, the amplitude of the carrier wave is maintained 
constant. An important feature of angle modulation is that it can provide better discrim- 
ination against noise and interference than amplitude modulation. As will be shown later 
in Section 2.7, however, this improvement in performance is achieved at the expense of 


increased transmission bandwidth; that is, angle modulation provides us with a Practical 


# Basic DEFINITIONS 


Let 6,(t) denote the angle of a modulated sinusoidal carrier, assumed to be a function of 
the message signal. We express the resulting angle-modulated wave as 


s(t) = A, cos[@,(t)] (2.19) 
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where A, is the carrier amplitude. A complete oscillation occurs whenever 6;(t) changes 
by 27 radians. If 6,(t) increases monotonically with time, the average frequency in Hertz, 
over an interval from ¢ to t + At, is given by 


_ 6{t + At) — lt) 
fat) = sae (2.20) 
We may thus define the instantaneous frequency of the angle-modulated signal s(t) as 
follows: 





. 6,(t + At) — 6,(t) 
J im | 2r At | (2.21) 
_ 1 dali) 
2m dt 


Thus, according to Equation (2.19), we may interpret the angle-modulated signal 
s(t) as a rotating phasor of length A. and angle 6,(t). The angular velocity of such a phasor 
is d6,(t)/dt measured in radians per second, in accordance with Equation (2.21). In the 
simple case of an unmodulated carrier, the angle 6;(t) is 


(t) = 2nft + Q- 


and the corresponding phasor rotates with a constant angular velocity equal to 2arf,. The 
constant @, is the value of 0;(t) at t = 0. i 
There are an infinite number of ways in which the angle 0,(t) may be varied in some 
manner with the message (baseband) signal. However, we shall consider only two com- 
monly used methods, phase modulation and frequency modulation, defined as follows: 


1. Phase modulation (PM) is that form of angle modulation in which the angle 0;(t) is 
varied linearly with the message signal m(t), as shown by 


a(t) = 2nf.t + kymlt) (2.22) 


The term 2f.t represents the angle of the unmodulated carrier; and the constant kp 
represents the phase sensitivity of the modulator, expressed in radians per volt on 
the assumption that m(t) is a voltage waveform. For convenience, we have assumed 
in Equation (2.22) that the angle of the unmodulated carrier is zero at £ = 0. The 
phase-modulated signal s(t) is thus described in the time domain by 


s(t) = A, cos[2af.t + kpm(t)] (2.23) 


2. Frequency modulation (FM) is that form of angle modulation in which the instan- 
taneous frequency f,(t) is varied linearly with the message signal m(t), as shown by 


filt) = fe + kemit) (2.24) 


The term f, represents the frequency of the unmodulated carrier, and the constant 
ky represents the frequency sensitivity of the modulator, expressed in Hertz per volt 
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FIGURE 2.20 [Illustrating the relationship between frequency modulation and phase modulation. 
(a) Scheme for generating an FM wave by using a phase modulator. (b) Scheme for generating a 
PM wave by using a frequency modulator. 


on the assumption that m(t) is a voltage waveform. Integrating Equation (2.24) with 
respect to time and multiplying the result by 27, we get 


0,(£) = 2af.t + 2k, Í m(t) dr (2.25) 


where, for convenience, we have assumed that the angle of the unmodulated carrier 
wave is zero at £ = 0. The frequency-modulated signal is therefore described in the 
time domain by 


s(t) = A, cos] 2 + ani |, m(7) a| (2.26) 


A consequence of allowing the angle 0;(t) to become dependent on the message signal 
m(t) as in Equation (2.22) or on its integral as in Equation (2.25) is that the zero crossings 
of a PM signal or FM signal no longer have a perfect regularity in their spacing; zero 
crossings refer to the instants of time at which a waveform changes from a negative to a 
positive value or vice versa. This is one important feature that distinguishes both PM and 
FM signals from an AM signal. Another important difference is that the envelope of a PM 
or FM signal is constant (equal to the carrier amplitude), whereas the envelope of an AM 
signal is dependent on the message signal. 

Comparing Equation (2.23) with (2.26) reveals that an FM signal may be regarded 
as a PM signal in which the modulating wave is fim(7) dz in place of m(t). This means 
that an FM signal can be generated by first integrating m(t) and then using the result as 
the input to a phase modulator, as in Figure 2.204. Conversely, a PM signal can be gen- 
erated by first differentiating m(t) and then using the result as the input to a frequency 
modulator, as in Figure 2.20b. We may thus deduce all the properties of PM signals from 
those of FM signals and vice versa. Henceforth, we concentrate our attention on FM 


signals. 


| 2.7 Frequency Modulation 





The FM signal s(t) defined by Equation (2.26) is a nonlinear function of the modulating 
signal m(t), which makes frequency modulation a nonlinear modulation process. Conse- 
quently, unlike amplitude modulation, the spectrum of an FM signal is not related in a 
simple manner to that of the modulating signal; rather, its analysis is much more difficult 
than that of an AM signal. 
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How then can we tackle the spectral analysis of an FM signal? We propose to provide 
an empirical answer to this important question by proceeding in the following manner: 


> We consider the simplest case possible, namely, that of a single-tone modulation that 
produces a narrowband FM signal. 

> We next consider the more general case also involving a single-tone modulation, but 
this time the FM signal is wideband. 


We could, of course, go on and consider the more elaborate case of a multitone FM signal, 
However, we propose not to do so, because our immediate objective is to establish an 
empirical relationship between the transmission bandwidth of an FM signal and the mes- 
sage bandwidth, As we shall subsequently see, the two-stage spectral analysis described 
here provides us with enough insight to propose a solution to the problem. 

Consider then a sinusoidal modulating signal defined by 


m(t) = Am cos(27f,,t) (2.27) 
The instantaneous frequency of the resulting FM signal equals 


f(t) = f + RpAm Cos(27f pt) 


= f, + Af cos(27f,,t) (2.28) 


where 
Af = kfm (2.29) 


The quantity Af is called the frequency deviation, representing the maximum departure 

of the instantaneous frequency of the FM signal from the carrier frequency f.. A funda- 

mental characteristic of an FM signal is that the frequency deviation Af is proportional 

to the amplitude of the modulating signal and is independent of the modulation frequency. 
Using Equation (2.28), the angle 0;(t) of the FM signal is obtained as 


b(t) = 2r Í filt) dr 
A (2.30) 
= laf.t + a sin(27f,,¢) 


The ratio of the frequency deviation Af to the modulation frequency f,,, is commonly called 
the modulation index of the FM signal. We denote it by B, and so write 


Af 
= 2.31 
B a (2.31) 
and 

6(t) = 2af.t + B sin(27f,t) (2.32) 


From Equation (2.32) we see that, in a physical sense, the parameter 8 represents the phase 
deviation of the FM signal, that is, the maximum departure of the angle @,(t) from the 
angle 27f,t of the unmodulated carrier; hence, 8 is measured in radians. 

The FM signal itself is given by 


s(t) = A, cos[2af,t + B sin(27f,,t)] (2.33) 
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Depending on the value of the modulation index B, we may distinguish two cases of 
frequency modulation: i 


> Narrowband FM, for which £ is small compared to one radian. 
> Wideband FM, for which £ is large compared to one radian. 


These two cases are considered next, in that order. 


a NARROWBAND FREQUENCY MODULATION 


Consider Equation (2.33), which defines an FM signa! resulting from the use of a sinusoidal 
modulating signal. Expanding this relation, we get 


s(t) = A, cos(27f,t) cos[B sin(27f,,t)] — A, sin(27f.t) sin[B sin(27f,,t)] (2.34) 


Assuming that the modulation index £ is small compared to one radian, we may use the 
following approximations: 


cos[8 sin(2mf,t)] = 1 
and ; 
sin[8 sin(27f,t)] = B sin(2mfx¢) 
Hence, Equation (2.34) simplifies to 
s(t) = A, cos(2af.t) — BA, sin(2nf.t) sin(2mft) (2.35) 


Equation (2.35) defines the approximate form of a narrowband FM signal produced by a 
sinusoidal modulating signal A,, cos(2 7f,,t). From this representation we deduce the mod- 
ulator shown in block diagram form in Figure 2.21. This modulator involves splitting the 
carrier wave A, cos(27f,t) into two paths. One path is direct; the other path contains a 
—90 degree phase-shifting network and a product modulator, the combination of which 
generates a DSB-SC modulated signal. The difference between these two signals produces 
a narrowband FM signal, but with some distortion. 

Ideally, an FM signal has a constant envelope and, for the case of a sinusoidal mod- 
ulating signal of frequency fm, the angle 6,(t) is also sinusoidal with the same frequency. 
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FIGURE 2.21 Block diagram of a method for generating a narrowband FM signal. 
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But the modulated signal produced by the narrowband modulator of Figure 2.21 differs 
from this ideal condition in two fundamental respects: 


1. The envelope contains a residual amplitude modulation and, therefore, varies with 
time. 

2. For a sinusoidal modulating wave, the angle 6,(¢) contains harmonic distortion in 
the form of third- and higher-order harmonics of the modulation frequency fn. 


However, by restricting the modulation index to 8 = 0.3 radians, the effects of residual 
AM and harmonic PM are limited to negligible levels. 
Returning to Equation (2.35), we may expand it as follows: 


s(t) = A, cos(27f.t) + 5 BAdcosl2a(f. + fat] — cos[2a(f. — fail} (2.36) 


This expression is somewhat similar to the corresponding one defining an AM signal, 
which is as follows: 


sam(t) = A, cos(2mf.t) + t pA{cos[2a(f. + f,)t] + cos[2a(f. = fn)él} (2.37) 


where u is the modulation factor of the AM signal. Comparing Equations (2.36) and 
(2.37), we see that in the case of sinusoidal modulation, the basic difference between an 
AM signal and a narrowband FM signal is that the algebraic sign of the lower side fre- 
quency in the narrowband FM is reversed. Thus, a narrowband FM signal requires essen- 
tially the same transmission bandwidth (i.e., 2f,,) as the AM signal. 
We may represent the narrowband FM signal with a phasor diagram as shown in 
Figure 2.22a, where we have used the carrier phasor as reference. We see that the resultant 
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FIGURE 2.22 A phasor comparison of narrowband FM and AM waves for sinusoidal modula- 
tion. (2) Narrowband FM wave. (b) AM wave. 
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of the two side-frequency phasors is always at right angles to the carrier phasor. The effect 
of this is to produce a resultant phasor representing the narrowband FM signal that is 
approximately of the same amplitude as the carrier phasor, but out of phase with respect 
to it. This phasor diagram should be contrasted with that of Figure 2.22b, representing 
an AM signal. In this latter case we see that the resultant phasor representing the AM 
signal has an amplitude that is different from that of the carrier phasor but always in phase 
with it. 


@ WIDEBAND FREQUENCY MODULATION 


We next wish to determine the spectrum of the single-tone FM signal of Equation (2.33) 
for an arbitrary value of the modulation index £. In general, an FM signal produced by a 
sinusoidal modulating signal, as in Equation (2.33), is in itself nonperiodic unless the 
carrier frequency f, is an integral multiple of the modulation frequency f,,. However, we 
may simplify matters by using the complex representation of band-pass signals described 
in Appendix 2. Specifically, we assume that the carrier frequency f, is large enough (com- 
pared to the bandwidth of the FM signal) to justify rewriting this equation in the form 


s(t) = Re[A, exp(j2af.t + jB sin(27f,,t))] 
= Re[S(t) exp( j2f,t)] 
where 5(t) is the complex envelope of the FM signal s(z), defined by 
S(t) = A, exp[jB sin(27f,,t)] (2.39) 


Thus, unlike the original FM signal s(t), the complex envelope &(t) is a periodic function 
of time with a fundamental frequency equal to the modulation frequency fn. We may 
therefore expand §(t) in the form of a complex Fourier series as follows: 


(2.38) 


s(t) = 5 c, EXP j2 TAfmnt) (2.40) 


where the complex Fourier coefficient c,, is defined by 


UF, 
Cn = fn | S(t) exp(—j27anf,,t) dt 


TA 
U2F m (2.41) 
= foe |, expliB sin 2rmfyt) — j2mnfpt] dt 
Define a new variable: 
x = Inf (2.42) 
Hence, we may rewrite Equation (2.41) in the new form 
Ca = £f exp|j(B sin x — nx)] dx (2.43) 


The integral on the right-hand side of Equation (2.43), except for a scaling factor, is 
recognized as the nth order Bessel function of the first kind? and argument B. This function 
is commonly denoted by the symbol J,(8), as shown by 


JB) = = | explilB sin x ~ na] dx (2.44) 
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Accordingly, we may reduce Equation (2.43) to 
on = A.J,AB) (2.45) 


Substituting Equation (2.45) in (2.40), we get, in terms of the Bessel function J,,(f), the 
following expansion for the complex envelope of the FM signal: 


x(t) =A, D JnlB) exp j271 ft) (2.46) 


n= 


Next, substituting Equation (2.46) in (2.38), we get 


s(t) = A,* Rel 5 JAB) exp[j2a(f. + nf (2.47) 


n=— o 


Interchanging the order of summation and evaluation of the real part in the right-hand 
side of Equation (2.47), we finally get 


st) = Ae D Ju(B) cosf2m(fe + mf, (2.48) 


This is the desired form for the Fourier series representation of the single-tone FM signal 
s(t) for an arbitrary value of B. The discrete spectrum of s(t) is obtained by taking the 
Fourier transforms of both sides of Equation (2.48); we thus have 


sip) = 42 È JDS- fe- nha) + BUF + f+ nfl (249) 


=—co 


In Figure 2.23 we have plotted the Bessel function J,,(8) versus the modulation index 
B for different positive integer values of n. We can develop further insight into the behavior 
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Jy(B) 





-0.2 








-0.4 
FIGURE 2.23 Plots of Bessel functions of the first kind for varying order. 
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of the Bessel function J,(8) by making use of the following properties (see Appendix 3 for 
more details): 


t JAB) = (-1)"J_,(B) for all #, both positive and negative (2.50) 


2. For small values of the modulation index 8, we have 


JB = 1 
Ji(B) = £ (2.51) 
JAB)~0, n>2 

3. È B=1 (2.52) 


Thus, using Equations (2.49)-(2.52) and the curves of Figure 2.23, we may make 
he following observations: 


1. The spectrum of an FM signal contains a carrier component and an infinite set of 
side frequencies located symmetrically on either side of the carrier at frequency sep- 
arations of fins 2fns 3fms*** - In this respect, the result is unlike that which prevails 
in an AM system, since in an AM system a sinusoidal modulating signal gives rise 
to only one pair of side frequencies. 

2. For the special case of 8 small compared with unity, only the Bessel coefficients Jo(8) 
and J,(8) have significant values, so that the FM signal is effectively composed of a 
carrier and a single pair of side frequencies at f, + fm. This situation corresponds to 
the special case of narrowband EM that was considered earlier. 

3. The amplitude of the carrier component varies with 8 according to Jo(8). That is, 
unlike an AM signal, the amplitude of the carrier component of an FM signal is 
dependent on the modulation index 8. The physical explanation for this property is 
that the envelope of an FM signal is constant, so that the average power of such a 
signal developed across a 1-ohm resistor is also constant, as shown by 

aly 
P 2 At (2.53) 
When the carrier is modulated to generate the FM signal, the power in the side 
frequencies may appear only at the expense of the power originally in the carrier, 
thereby making the amplitude of the carrier component dependent on £. Note that 
the average power of an FM signal may also be determined from Equation (2.48), 
obtaining 
P=542 © 7a) (2.54) 
Substituting Equation (2.52) into (2.54), the expression for the average power P 
reduces to Equation (2.53), and so it should. 


> EXAMPLE 2.2 


In this example, we wish to investigate the ways in which variations in the amplitude and 
frequency of a sinusoidal modulating signal affect the spectrum of the FM signal. Consider 
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FIGURE 2.24 Discrete amplitude spectra of an FM signal, normalized with respect to the carrier 
amplitude, for the case of sinusoidal modulation of fixed frequency and varying amplitude. Only 
the spectra for positive frequencies are shown. 





first the case when the frequency of the modulating signal is fixed, but its amplitude is varied, 
producing a corresponding variation in the frequency deviation Af. Thus, keeping the mod- 
ulation frequency f» fixed, we find that the amplitude spectrum of the resulting FM signal is 
as shown plotted in Figure 2.24 for B = 1, 2, and 5. In this diagram we have normalized the 
spectrum with respect to the unmodulated carrier amplitude. 

Consider next the case when the amplitude of the modulating signal is fixed; that is, the 
frequency deviation Af is maintained constant, and the modulation frequency fy, is varied. 
In this case we find that the amplitude spectrum of the resulting FM signal is as shown 
plotted in Figure 2.25 for 8 = 1, 2, and 5. We see that when Af is fixed and £ is increased, 
we have an increasing number of spectral lines crowding into the fixed frequency interval 
fe - Af <|f |< f+ Af. That is, when 8 approaches infinity, the bandwidth of the FM wave 
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FiGuRE 2.25 Discrete amplitude spectra of an FM signal, normalized with respect to the carrier 
amplitude, for the case of sinusoidal modulation of varying frequency and fixed amplitude. Only 
the spectra for positive frequencies are shown. 





approaches the limiting value of 2Af, which is an important point to keep in mind for later 
discussion. 


a TRANSMISSION BANDWIDTH OF FM SIGNALS 


In theory, an FM signal contains an infinite number of side frequencies so that the band- 
width required to transmit such a signal is similarly infinite in extent. In practice, however, 
we find that the FM signal is effectively limited to a finite number of significant side 
frequencies compatible with a specified amount of distortion. We may therefore specify 
an effective bandwidth required for the transmission of an FM signal. Consider first the 
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case of an FM signal generated by a single-tone modulating wave of frequency fn- In such 
an FM signal, the side frequencies that are separated from the carrier frequency f; by an 
amount greater than the frequency deviation Af decrease rapidly toward zero, so that the 
bandwidth always exceeds the total frequency excursion, but nevertheless is limited. Spe- 
cifically, for large values of the modulation index £, the bandwidth approaches, and is 
only slightly greater than, the total frequency excursion 2Af in accordance with the situ- 
ation shown in Figure 2.25. On the other hand, for small values of the modulation index 
B, the spectrum of the FM signal is effectively limited to the carrier frequency f; and one 
pair of side frequencies at fe + fins SO that the bandwidth approaches 2f,,. We may thus 
define an approximate rule for the transmission bandwidth of an FM signal generated by 
a single-tone modulating signal of frequency fm as follows: 


Br = 2Af + 2fy = aas(t + z) (2.55) 


This empirical relation is known as Carson’s rule.* 

For an alternative assessment of the bandwidth requirement of an FM signal, we 
may use a definition based on retaining the maximum number of significant side frequen- 
cies whose amplitudes are all greater than some selected value. A convenient choice for 
this value is 1 percent of the unmodulated carrier amplitude. We may thus define the 
transmission bandwidth of an FM wave as the separation between the two frequencies 
beyond which none of the side frequencies is greater than 1 percent of the carrier amplitude 
obtained when the modulation is removed. That is, we define the transmission bandwidth 
as 2Mmaxfns Where fn is the modulation frequency and nmax is the largest value of the 
integer 7 that satisfies the requirement \J,(B) | > 0.01. The value of nmax varies with the 
modulation index ß and can be determined readily from tabulated values of the Bessel 
function J„(8). Table 2.2 shows the total number of significant side frequencies (including 
both the upper and lower side frequencies) for different values of £, calculated on the 1 
percent basis explained herein. The transmission bandwidth By calculated using this pro- 
cedure can be presented in the form of a universal curve by normalizing it with respect to 
the frequency deviation Af and then plotting it versus B. This curve is shown in Figure 
2.26, which is drawn as a best fit through the set of points obtained by using Table 2.2. 
In Figure 2.26 we note that as the modulation index B is increased, the bandwidth occupied 


TABLE 2.2 Number of significant side 
frequencies of a wideband FM signal for varying 
modulation index 













Modulation Index Number of Significant Side Frequencies 

B 2tnax 
0.1 2 
0.3 4 
0.5 4 
1.0 ` 6 
2.0 8 
5.0 16 

10.0 28 

20.0 50 


30.0 70 
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FIGURE 2.26 Universal curve for evaluating the 1 percent bandwidth of an FM wave. 


by the significant side frequencies drops toward that over which the carrier frequency 
actually deviates. This means that small values of the modulation index are relatively 
more extravagant in transmission bandwidth than are the larger values of 8. 

Consider next the more general case of an arbitrary modulating signal m(t) with its 
highest frequency component denoted by W. The bandwidth required to transmit an 
FM signal generated by this modulating signal is estimated by using a worst-case tone- 
modulation analysis. Specifically, we first determine the so-called deviation ratio D, defined 
as the ratio of the frequency deviation Af, which corresponds to the maximum possible 
amplitude of the modulation signal m(t), to the highest modulation frequency W; these 
conditions represent the extreme cases possible. The deviation ratio D plays the same role 
for nonsinusoidal modulation that the modulation index B plays for the case of sinusoidal 
modulation. Then, replacing 8 by D and replacing f,, with W, we may use Carson’s rule 
given by Equation (2.55) or the universal curve of Figure 2.26 to obtain a value for the 
transmission bandwidth of the FM signal. From a practical viewpoint, Carson’s rule some- 
what underestimates the bandwidth requirement of an FM system, whereas using the uni- 
versal curve of Figure 2.26 yields a somewhat conservative result. Thus, the choice of a 
transmission bandwidth that lies between the bounds provided by these two rules of thumb 
is acceptable for most practical purposes. 


B EXAMPLE 2.3 


In North America, the maximum value of frequency deviation Af is fixed at 75 kHz for 
commercial FM broadcasting by radio. If we take the modulation frequency W = 15 kHz, 
which is typically the “maximum” audio frequency of interest in FM transmission, we find 
that the corresponding value of the deviation ratio is 


D=—== 
15 


Using Carson’s rule of Equation (2.55), replacing B by D, and replacing fm by W, the ap- 
proximate value of the transmission bandwidth of the FM signal is obtained as 


By = 2(75 + 15) = 180 kHz 
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On the other hand, use of the curve of Figure 2.26 gives the transmission bandwidth of the 
FM signal to be 


By = 3.2Af = 3.2 x 75 = 240 kHz 


In practice, a bandwidth of 200 kHz is allocated to each FM transmitter. On this basis, 
Carson’s rule underestimates the transmission bandwidth by 10 percent, whereas the universa] 
curve of Figure 2.26 overestimates it by 20 percent. <q 


= GENERATION OF FM SIGNALS 


There are essentially two basic methods of generating frequency-modulated signals, 
namely, direct FM and indirect FM. In the direct method the carrier frequency is directly 
varied in accordance with the input baseband signal, which is readily accomplished using 
a voltage-controlled oscillator. In the indirect method, the modulating signal is first used 
to produce a narrowband FM signal, and frequency multiplication is next used to increase 
the frequency deviation to the desired level. The indirect method is the preferred choice 
for frequency modulation when the stability of carrier frequency is of major concern as in 
commercial radio broadcasting, as described next. 


Indirect FM? 


A simplified block diagram of an indirect FM system is shown in Figure 2.27. The 
message (baseband) signal m(t) is first integrated and then used to phase-modulate a 
crystal-controlled oscillator; the use of crystal control provides frequency stability. To 
minimize the distortion inherent in the phase modulator, the maximum phase deviation 
or modulation index £ is kept small, thereby resulting in a narrowband FM signal; for the 
implementation of the narrow-band phase modulator, we may use the arrangement de- 
scribed in Figure 2.21, The narrowband FM signal is next multiplied in frequency by means 
of a frequency multiplier so as to produce the desired wideband FM signal. 

A frequency multiplier consists of a nonlinear device followed by a band-pass filter, 
as shown in Figure 2.28. The implication of the nonlinear device being memoryless is that 
it has no energy-storage elements. The input-output relation of such a device may be 
expressed in the general form 


v(t) = ays(t) + as? (t) + +++ + a,s"(t) (2.56) 


where 41, do,..., 4, are coefficients determined by the operating point of the device, and 
n is the highest order of nonlinearity. In other words, the memoryless nonlinear device is 
an nth power-law device. The input s(t) is an FM signal defined by 


s(t) = A, cos| Det + amk, | m(7) a 
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FIGURE 2.27 Block diagram of the indirect method of generating a wideband FM signal. 
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FIGURE 2.28 Block diagram of frequency multiplier. 


whose instantaneous frequency is y 
Silt) = f + Rym(t) (2.57) 


The mid-band frequency of the band-pass filter in Figure 2.28 is set equal to #f,, where f, 
is the carrier frequency of the incoming FM signal s(t). Moreover, the band-pass filter is 
designed to have a bandwidth equal to # times the transmission bandwidth of s(t). In 
Section 2.8 dealing with nonlinear effects in FM systems, we describe the spectral contri- 
butions of such nonlinear terms as the second- and third-order terms in the input-output 
relation of Equation (2.56). For now it suffices to say that after band-pass filtering of the 
nonlinear device’s output v(t), we have a new FM signal defined by 


s(t) = Al cos| Dnt + 2ank, Í m(7) a| (2.58) 


whose instantaneous frequency is 
fit) = nf. + nket) (2.59) 


Thus, comparing Equation (2.59) with (2.57), we see that the nonlinear processing circuit 
of Figure 2.28 acts as a frequency multiplier. The frequency multiplication ratio is deter- 
mined by the highest power n in the input-output relation of Equation (2.56), character- 
izing the memoryless nonlinear device. 


& DEMODULATION OF FM SIGNALS 


Frequency demodulation is the process that enables us to recover the original modulating 
signal from a frequency-modulated signal. The objective is to produce a transfer charac- 
teristic that is the inverse of that of the frequency modulator, which can be realized directly 
or indirectly. Here we describe a direct method of frequency demodulation involving the 
use of a popular device known as a frequency discriminator, whose instantaneous output 
amplitude is directly proportional to the instantaneous frequency of the input FM signal. 
In Section 2.14, we describe an indirect method of frequency demodulation that uses 
another popular device known as a phase-locked loop. 

Basically, the frequency discriminator consists of a slope circuit followed by an en- 
velope detector. An ideal slope circuit is characterized by a frequency response that is 
purely imaginary, varying linearly with frequency inside a prescribed frequency interval. 
Consider the frequency response depicted in Figure 2.294, which is defined by 
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H,(f) = jana( 1 ie Ba), pe Be Br (2.60) 
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FIGURE 2.29 (a) Frequency response of ideal slope circuit. (b) Frequency response of the slope 
circuit’s complex low-pass equivalent. (c) Frequency response of the ideal slope circuit comple- 
mentary to that of part (a). 


where a is a constant. We wish to evaluate the response of this slope circuit, denoted by 
sı(t), which is produced by an FM signal s(t) of carrier frequency f, and transmission 
bandwidth Br. It is assumed that the spectrum of s(t) is essentially zero outside the fre- 
quency interval f, — B2 < | f| = fs + Br/2. 

We may simplify the analysis of the frequency discriminator by invoking the iso- 
morphism between a real-valued band-pass filter and a corresponding complex-valued 
low-pass filter. This isomorphism is discussed in Appendix 2. According to the material 
presented in that appendix, we may replace the band-pass filter with frequency response 
H,(f) with an equivalent low-pass filter with frequency response H,(f) by doing two 
things: 


1. We shift H,(f) to the right by f., where f, is the midband frequency of the band- 
pass filter; this operation aligns the translated frequency response of the equivalent 
low-pass filter with that of the band-pass filter. 


2. We set A, (f — f.) equal to 2H,(f) for f > 0. 
Thus for the problem at hand we get 
Alf - f) =2H(f), f>0 (2.61) 
Hence, using Equations (2.60) and (2.61), we get 





m : Br Br Br 
FL(f) = jal 7 i 2), 25157 (2.62) 
0, elsewhere 


which is plotted in Figure 2.29b. 
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The incoming FM signal s(t) is defined by Equation (2.26), which is reproduced here 
for convenience: 


s(t) = A, cos| Det + 2ak, Í miT) a- 


Given that the carrier frequency f, is high compared to the transmission bandwidth of the 
FM signal s(t), the complex envelope of s(t) is 


a(t) = A, exp| iy Í m7) a- (2.63) 


Let §,(¢) denote the complex envelope of the response of the slope circuit defined by 
Figure 2.294 due to s(t). Then, following the material presented in Appendix 2, we may 
express the Fourier transform of §,(t) as follows: 


Sif) = 5 APSA) 
(2.64) 
Br\ « Br B 
paral f i r\s1p, -7 shez 
0, elsewhere 


where S(f) is the Fourier transform of 3(t). From Fourier analysis we know that multipli- 
cation of the Fourier transform of a signal by j2af is equivalent to differentiating the 
signal in the time domain; see item 8 of Table A6.2. Hence, from Equation (2.64) we 
deduce 


Z(t) = “| + jaBzilt J (2.65) 
Substituting Equation (2.63) into (2.65), we get 
§,(t) = jnB0A,|1 + 2 mt expl fant, | mlr) ar] (2.66) 
By o 


The desired response of the slope circuit is therefore 


silt) = Refs (t) exp( j2mf-t)] 

2k; ‘ | (267) 

= nmByaå.| 1 + Be mit) | cos] 2af,t + 2k, if mirt) dr + p 

The signal s,(t) is a hybrid-modulated signal in which both amplitude and frequency of 
the carrier wave vary with the message signal m(t). However, provided that we choose 
2k 
Br 
then we may use an envelope detector to recover the amplitude variations and thus, except 
for a bias term, obtain the original message signal. The resulting envelope-detector output 
is therefore 


<1 for all t 





m(t) 





|&(t)] = Brad 1 + E min | (2.68) 


The bias term 7B7aA, in the right-hand side of Equation (2.68) is proportional to 
the slope 4 of the transfer function of the slope circuit. This suggests that the bias may be 
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FIGURE 2.30 Block diagram of frequency discriminator. 


removed by subtracting from the envelope-detector output | §,(t)| the output of a second 
envelope detector preceded by the complementary slope circuit with a frequency response 
H,(f) as described in Figure 2.29c. That is, the two slope circuits are related by 


Af) = Al- f) (2.69) 


Let s(t) denote the response of the complementary slope circuit produced by the incoming 
FM signal s(t). Then, following a procedure similar to that just described, we may write 


|5,(¢)| = wByaA,|1 - = mt (2.70) 
T 


where §,(t) is the complex envelope of the signal s2(t). The difference between the two 


envelopes in Equations (2.68) and (2.70) is 


solt) = |5,(4)| — |52(2)| 


= Amk,aAm(t) a 


which is a scaled version of the original message signal m(t) and free from bias. 

We may thus model the ideal frequency discriminator as a pair of slope circuits with 
their complex transfer functions related by Equation (2.69), followed by envelope detectors 
and finally a summer, as in Figure 2.30. This scheme is called a balanced frequency 
discriminator. 


FM STEREO MULTIPLEXING® 


Stereo multiplexing is a form of frequency-division multiplexing (FDM) designed to trans- 
mit two separate signals via the same carrier. It is widely used in FM radio broadcasting 
to send two different elements of a program (e.g., two different sections of an orchestra, 
a vocalist and an accompanist) so as to give a spatial dimension to its perception by a 
listener at the receiving end. 

The specification of standards for FM stereo transmission is influenced by two 
factors: 


1. The transmission has to operate within the allocated FM broadcast channels. 
2. It has to be compatible with monophonic radio receivers. 


The first requirement sets the permissible frequency parameters, including frequency de- 
viation. The second requirement constrains the way in which the transmitted signal is 
configured. 

Figure 2.314 shows the block diagram of the multiplexing system used in an FM 
stereo transmitter. Let m(t) and m,(t) denote the signals picked up by left-hand and 
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FIGURE 2.31 (a) Multiplexer in transmitter of FM stereo. (b) Demultiplexer in receiver of FM 
stereo. 


right-hand microphones at the transmitting end of the system. They are applied to a 
simple matrixer that generates the sum signal, m(t) + m,(t), and the difference signal, 
m(t) — m,(t). The sum signal is left unprocessed in its baseband form; it is available for 
monophonic reception. The difference signal and a 38-kHz subcarrier (derived from a 19- 
kHz crystal oscillator by frequency doubling) are applied to a product modulator, thereby 
producing a DSB-SC modulated wave. In addition to the sum signal and this DSB-SC 
modulated wave, the multiplexed signal m(t) also includes a 19-kHz pilot to provide a 
reference for the coherent detection of the difference signal at the stereo receiver. Thus the 
multiplexed signal is described by 


m(t) = [m(t) + m,(t)] + [Pr — m,(t)] cos(4af.t) + K cos(27f.t) (2.72) 


where f, = 19 kHz, and K is the amplitude of the pilot tone. The multiplexed signal m/(t) 
then frequency-modulates the main carrier to produce the transmitted signal. The pilot is 
allotted between 8 and 10 percent of the peak frequency deviation; the amplitude K in 
Equation (2.72) is chosen to satisfy this requirement. 

At a stereo receiver, the multiplexed signal m(t) is recovered by frequency demodu- 
lating the incoming FM wave. Then m(t) is applied to the demultiplexing system shown 
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in Figure 2.31b. The individual components of the multiplexed signal m(t) are separated 
by the use of three appropriate filters. The recovered pilot (using a narrowband filter tuned 
to 19 kHz) is frequency doubled to produce the desired 38-kHz subcarrier. The availability 
of this subcarrier enables the coherent detection of the DSB-SC modulated wave, thereby 
recovering the difference signal, #2,(t) — m,(t). The baseband low-pass filter in the top 
path of Figure 2.310 is designed to pass the sum signal, m(t) + ,(¢). Finally, the simple 
matrixer reconstructs the left-hand signal m(t) and right-hand signal m,(t), except for 
scaling factors, and applies them to their respective speakers. 


| 2.8 Nonlinear Effects in FM Systems 


In the preceding two sections, we studied frequency modulation theory and methods for 
its generation and demodulation. We complete the discussion of frequency modulation by 
considering nonlinear effects in FM systems. 

Nonlinearities, in one form or another, are present in all electrical networks. There 
are two basic forms of nonlinearity to consider: 


1. The nonlinearity is said to be strong when it is introduced intentionally and in a 
controlled manner for some specific application. Examples of strong nonlinearity 
include square-law modulators, hard-limiters, and frequency multipliers. 

2. The nonlinearity is said to be weak when a linear performance is desired, but. non- 
linearities of a parasitic nature arise due to imperfections. The effect of such weak 
nonlinearities is to limit the useful signal levels in a system and thereby become an 
important design consideration. 


In this section we examine the effects of weak nonlinearities on frequency modulation. 
Consider a communications channel, the transfer characteristic of which is defined 
by the nonlinear input-output relation 


v(t) = ayu,(t) + azu?(t) + asv7(t) (2.73) 


where y(t) and v,(t) are the input and output signals, respectively, and a, 42, and a; are 
constants; Equation (2.73) is a truncated version of Equation (2.56) used in the context 
of frequency multiplication. The channel described in Equation (2.73) is said to be me- 
moryless in that the output signal v,(t) is an instantaneous function of the input signal 
v,(t) (i.e., there is no energy storage involved in the description). We wish to determine the 
effect of transmitting a frequency-modulated wave through such a channel. The FM signal 
is defined by 


u(t) = A, cos[2af.t + (2)] 


where 


o(t).= 27k; | mit) dr 


For this input signal, the use of Equation (2.73) yields 


v(t) = aÀ, cos[2af.t + b(t)] + aA? cos*[2af.t + (t)] 


4 
+ a3A3 cos*[2af.t + $(t)] A 
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Expanding the squared and cubed cosine terms in Equation (2.74) and then collecting 
common terms, we get 


v(t) = Zaa? + (aa, + 2 aa?) cos[2af.t + &(2)] 
+ iaa? cos[4af.t + 2¢(t)] (2.75) 


+ iana cos[6rf.t + 3¢(t)] 


Thus the channel output consists of a DC component and three frequency-modulated 
signals with carrier frequencies f., 2f., and 3f,; the latter components are contributed by 
the linear, second-order, and third-order terms of Equation (2.73), respectively. 

To extract the desired FM signal from the channel output v,(z), that is, the particular 
component with carrier frequency, f,, it is necessary to separate the FM signal with this 
carrier frequency from the one with the closest carrier frequency, 2f,. Let Af denote the 
frequency deviation of the incoming FM signal v,(t), and W denote the highest frequency 
component of the message signal m(t). Then, applying Carson’s rule and noting that the 
frequency deviation about the second harmonic of the carrier frequency is doubled, we 
find that the necessary condition for separating the desired FM signal with the carrier 
frequency f, from that with the carrier frequency 2f, is 


2f,- (2Af + W)>f+af+w 
or 
f: > 3Af + 2W (2.76) 


Thus, by using a band-pass filter of midband frequency f, and bandwidth 2Af + 2W, the 
channel output is reduced to 


ult) = (aa. + 3 aa) cos[2rf.t + (i) (2.77) 


We see therefore that the only effect of passing an FM signal through a channel with 
amplitude nonlinearities, followed by appropriate filtering, is simply to modify its ampli- 
tude. That is, unlike amplitude modulation, frequency modulation is not affected by dis- 
tortion produced by transmission through a channel with amplitude nonlinearities. It is 
for this reason that we find frequency modulation used in microwave radio systems: It 
permits the use of highly nonlinear amplifiers and power transmitters, which are particu- 
larly important to producing a maximum power output at radio frequencies. 

An FM system is extremely sensitive to phase nonlinearities, however, as we would 
intuitively expect. A common type of phase nonlinearity that is encountered in microwave 
radio systems is known as AM-to-PM conversion. This is the result of the phase charac- 
teristic of repeaters or amplifiers used in the system being dependent on the instantaneous 
amplitude of the input signal. In practice, AM-to-PM conversion is characterized by a 
constant K, which is measured in degrees per dB and may be interpreted as the peak phase 
change at the output for a 1-dB change in envelope at the input. When an FM wave is 
transmitted through a microwave radio link, it picks up spurious amplitude variations due 
to noise and interference during the course of transmission, and when such an FM wave 
is passed through a repeater with AM-to-PM conversion, the output will contain unwanted 
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phase modulation and resultant distortion. It is therefore important to keep the AM-to. 
PM conversion at a low level. For example, for a good microwave repeater, the AM-to. 
PM conversion constant K is less than 2 degrees per dB. 


| 2.9 Swuperheterodyne Receiver’ 


In a broadcasting system, irrespective of whether it is based on amplitude modulation or 
frequency modulation, the receiver not only has the task of demodulating the incoming 
modulated signal, but it is also required to perform some other system functions: 


© Carrier-frequency tuning, the purpose of which is to select the desired signal (i.e, 
desired radio or TV station). 

» Filtering, which is required to separate the desired signal from other modulated sig- 
nals that may be picked up along the way. 

> Amplification, which is intended to compensate for the loss of signal power incurred 
in the course of transmission. 


The superheterodyne receiver, or superhet as it is often referred to, is a special type of 
receiver that fulfills all three functions, particularly the first two, in an elegant and practical 
fashion. Specifically, it overcomes the difficulty of having to build a tunable highly selective 
and variable filter. Indeed, practically all radio and TV receivers now being made are of 
the superheterodyne type. 

Basically, the receiver consists of a radio-frequency (RF) section, a mixer and local 
oscillator, an intermediate-frequency (IF) section, demodulator, and power amplifier. Typ- 
ical frequency parameters of commercial AM and FM radio receivers are listed in Table 
2.3. Figure 2.32 shows the block diagram of a superheterodyne receiver for amplitude 
modulation using an envelope detector for demodulation. 

The incoming amplitude-modulated wave is picked up by the receiving antenna and 
amplified in the RF section that is tuned to the carrier frequency of the incoming wave. 
The combination of mixer and local oscillator (of adjustable frequency) provides a het- 
erodyning function, whereby the incoming signal is converted to a predetermined fixed 
intermediate frequency, usually lower than the incoming carrier frequency. This frequency 
translation is achieved without disturbing the relation of the sidebands to the carrier; see 
Section 2.4. The result of the heterodyning is to produce an intermediate-frequency carrier 


defined by 
fir = fio ~ frr (2.78) 


where fio is the frequency of the local oscillator and fp is the carrier frequency of the 
incoming RF signal. We refer to fy as the intermediate frequency (IF), because the signal 


TABLE 2.3 Typical frequency parameters of AM and FM 
radio receivers 





AM Radio FM Radio 
RF carrier range 0.535-1.605 MHz 88-108 MHz 
Midband frequency of IF section 0.455 MHz 10.7 MHz 


TF bandwidth 10 kHz 200 kHz 
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FIGURE 2.32 Basic elements of an AM radio receiver of the superheterodyne type. 


is neither at the original input frequency nor at the final baseband frequency. The mixer— 
local oscillator combination is sometimes referred to as the first detector, in which case 
the demodulator is called the second detector. 

The IF section consists of one or more stages of tuned amplification, with a band- 
width corresponding to that required for the particular type of modulation that the receiver 
is intended to handle. The IF section provides most of the amplification and selectivity in 
the receiver. The output of the IF section is applied to a demodulator, the purpose of which 
is to recover the baseband signal. If coherent detection is used, then a coherent signal 
source must be provided in the receiver. The final operation in the receiver is the power 
amplification of the recovered message signal. 

In a superheterodyne receiver the mixer will develop an intermediate frequency out- 
put when the input signal frequency is greater or less than the local oscillator frequency 
by an amount equal to the intermediate frequency. That is, there are two input frequencies, 
namely, | fro + fr|, which will result in fw at the mixer output. This introduces the 
possibility of simultaneous reception of two signals differing in frequency by twice the 
intermediate frequency. For example, a receiver tuned to 0.65 MHz and having an IF of 
0.455 MHz is subject to an image interference at 1.56 MHz; indeed, any receiver with this 
value of IF, when tuned to any station, is subject to image interference at a frequency of 
0.910 MHz higher than the desired station. Since the function of the mixer is to produce 
the difference between two applied frequencies, it is incapable of distinguishing between 
the desired signal and its image in that it produces an IF output from either one of them. 
The only practical cure for image interference is to employ highly selective stages in the 
RF section (i.e., between the antenna and the mixer) in order to favor the desired signal 
and discriminate against the undesired or image signal. The effectiveness of suppressing 
unwanted image signals increases as the number of selective stages in the RF section in- 
creases, and as the ratio of intermediate to signal frequency increases. 

The basic difference between AM and FM superheterodyne receivers lies in the use 
of an FM demodulator such as limiter-frequency discriminator. In an FM system, the 
message information is transmitted by variations of the instantaneous frequency of a 
sinusoidal carrier wave, and its amplitude is maintained constant. Therefore, any varia- 
tions of the carrier amplitude at the receiver input must result from noise or interference. 
An amplitude limiter, following the IF section, is used to remove amplitude variations 
by clipping the modulated wave at the IF section output almost to the zero axis. The 
resulting rectangular wave is rounded off by a band-pass filter that suppresses har- 
monics of the carrier frequency. Thus the filter output is again sinusoidal, with an ampli- 
tude that is practically independent of the carrier amplitude at the receiver input (see 
Problem 2.42). 
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| 2.10 Noise in CW Modulation Systems 





Up to this point in our discussion we have focused attention on the characterization of 
continuous-wave (CW) modulation techniques, entirely from a deterministic perspective, 
In the remainder of the chapter, we study the effects of channel noise on the reception of 
CW modulated signals and thereby develop a deeper understanding of the behavior of 
analog communications. 

To undertake such a study we follow the customary practice by formulating two 
models: 


1. Channel model, which assumes a communication channel that is distortionless bur 
perturbed by additive white Gaussian noise (AWGN). 

2. Receiver model, which assumes a receiver consisting of an ideal band-pass filter fol- 
lowed by an ideal demodulator appropriate for the application at hand; the band- 
pass filter is used to minimize the effect of channel noise. 


These simplifying assumptions are made in order to obtain a basic understanding of the 
way in which noise affects the performance of the receiver. Moreover, they provide a 
framework for the comparison of different CW modulation-demodulation schemes. 

Figure 2.33 shows the noisy receiver model that combines the above two assump- 
tions. In this figure, s(t) denotes the incoming modulated signal and w(t) denotes the 
channel noise. The received signal is therefore made up of the sum of s(t) and w(t); this is 
the signal that the receiver has to work on. The band-pass filter in the model of Figure 
2.33 represents the combined filtering action of the tuned amplifiers used in the actual 
receiver for the purpose of signal amplification prior to demodulation. The bandwidth of 
this band-pass filter is just wide enough to pass the modulated signal s(t) without distor- 
tion. As for the demodulator in the model of Figure 2.33, its details naturally depend on 
the type of modulation used. 


E SIGNAL-TO-NOISE RATIOS: BASIC DEFINITIONS 


Let the power spectral density of the noise w(t) be denoted by No/2, defined for both 
positive and negative frequencies; that is, No is the average noise power per unit bandwidth 
measured at the front end of the receiver. We also assume that the band-pass filter in the 
receiver model of Figure 2.33 is ideal, having a bandwidth equal to the transmission band- 
width By of the modulated signal s(t) and a midband frequency equal to the carrier fre- 
quency f. The latter assumption is justified for double sideband—suppressed carrier (DSB- 
SC) modulation, full amplitude modulation (AM), and frequency modulation (FM); the 
cases of single sideband (SSB) modulation and vestigial sideband (VSB) modulation require 
special considerations. Taking the midband frequency of the band-pass filter to be the 
same as the carrier frequency f., we may model the power spectral density Sx(f) of the 
noise x(t), resulting from the passage of the white noise w(t) through the filter, as shown 
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FIGURE 2.33 Receiver model. 


2-10 Noise in CW Modulation Systems 131 


in Figure 2.34. Typically, the carrier frequency f, is large compared to the transmission 
bandwidth By. We may therefore treat the filtered noise n(t) as a narrowband noise rep- 
resented in the canonical form 


n(t) = a,(t) cos(2af.t) — mo(t) sin(2zf,t) (2.79) 


where #;(t) is the in-phase noise component and no(t) is the quadrature noise component, 
both measured with respect to the carrier wave A, cos(27f,t). The filtered signal x(t) avail- 
able for demodulation is defined by i 


x(t) = s(t) + n(t) (2.80) 


The details of s(t) depend on the type of modulation used. In any event, the average noise 
power at the demodulator input is equal to the total area under the curve of the power 
spectral density Sn(f). From Figure 2.34 we readily see that this average noise power is 
equal to NoBy. Given the format of s(t), we may also determine the average signal power 
at the demodulator input. With the demodulated signal s(t) and the filtered noise (z(t) 
appearing additively at the demodulator input in accordance with Equation (2.80), we 
may go on to define an input signal-to-noise ratio, (SNR);, as the ratio of the average 
power of the modulated signal s(t) to the average power of the filtered noise n(t). 

A more useful measure of noise performance, however, is the output signal-to-noise 
ratio, (SNR)o, defined as the ratio of the average power of the demodulated message signal 
to the average power of the noise, both measured at the receiver output. The output signal- 
to-noise ratio provides an intuitive measure for describing the fidelity with which the de- 
modulation process in the receiver recovers the message signal from the modulated signal 
in the presence of additive noise. For such a criterion to be well defined, the recovered 
message signal and the corruptive noise component must appear additively at the demod- 
ulator output. This condition is perfectly valid in the case of a receiver using coherent 
detection. On the other hand, when the receiver uses envelope detection as in full AM or 
frequency discrimination as in FM, we have to assume that the average power of the filtered 
noise n(t) is relatively low to justify the use of output signal-to-noise ratio as a measure of 
receiver performance. 

The output signal-to-noise ratio depends, among other factors, on the type of mod- 
ulation used in the transmitter and the type of demodulation used in the receiver. Thus it 
is informative to compare the output signal-to-noise ratios for different modulation- 
demodulation systems. However, for this comparison to be of meaningful value, it must 
be made on an equal basis as described here: 


e The modulated signal s(t) transmitted by each system has the same average power. 


> The channel noise w(t) has the same average power measured in the message band- 
width W. 





Te ie) fe 


FIGURE 2.34 Idealized characteristic of band-pass filtered noise. 
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FIGURE 2.35 The baseband transmission model, assuming a message signal of bandwidth W, 
used for calculating the channel signal-to-noise ratio. 


Accordingly, as a frame of reference we define the channel signal-to-noise ratio, (SNR)c, 
as the ratio of the average power of the modulated signal to the average power of channel 
noise in the message bandwidth, both measured at the receiver input. This definition is 
illustrated in Figure 2.35. 

For the purpose of comparing different continuous-wave (CW) modulation systems, 
we normalize the receiver performance by dividing the output signal-to-noise ratio by the 
channel signal-to-noise ratio. We thus define a figure of merit for the receiver as follows: 

. .. _ (SNR)o 
Figure of merit (SNR)c (2.81) 
Clearly, the higher the value of the figure of merit, the better will the noise performance 
of the receiver be. The figure.of merit may equal one, be less than one, or be greater than 
one, depending on the type of modulation used, which will become apparent from the 
discussion that follows. 


2.11 Noise in Linear Receivers 
Using Coherent Detection 


From Sections 2.2 and 2.3 we recall that the demodulation of an amplitude-modulated 
wave depends on whether the carrier is suppressed or not. When the carrier is suppressed 
we usually require the use of coherent detection, in which case the receiver is linear. On 
the other hand, when the amplitude modulation includes transmission of the carrier, de- 
modulation is accomplished simply by using an envelope detector, in which case the re- 
ceiver is nonlinear. In this section we study the effect of noise on the performance of a 
linear receiver. The more difficult case of a nonlinear receiver is deferred to Section 2.12. 

Consider the case of DSB-SC modulation Figure 2.36 shows the model of a DSB-SC 
receiver using a coherent detector. The use of coherent detection requires multiplication 
of the filtered signal x(t) by a locally generated sinusoidal wave cos(27f,t) and then low- 
pass filtering the product. To simplify the analysis, we assume that the amplitude of the 
locally generated sinusoidal wave is unity. For this demodulation scheme to operate sat- 
isfactorily, however, it is necessary that the local oscillator be synchronized both in phase 
and in frequency with the oscillator generating the carrier wave in the transmitter. We 
assume that this synchronization has been achieved. 

The DSB-SC component of the filtered signal x(t) is expressed as 


s(t) = CA, cos(27f,t)m(t) (2.82) 


where A, cos(27f,t) is the sinusoidal carrier wave and m(t) is the message signal. In the 
expression for s(t) in Equation (2.82) we have included a system-dependent scaling factor 
C, the purpose of which is to ensure that the signal component s(¢) is measured in the same 
units as the additive noise component x(t). We assume that m{(ż) is the sample function of 
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FIGURE 2.36 Model of DSB-SC receiver using cohérent detection. 


a stationary process of zero mean, whose power spectral density Sy(f) is limited to a 
maximum frequency W; that is, W is the message bandwidth. The average power P of the 
message signal is the total area under the curve of power spectral density, as shown by 


w 
P= | Suf) df (2.83) 


The carrier wave is statistically independent of the message signal. To emphasize this 
independence, the carrier should include a random phase that is uniformly distributed over 
27 radians. In the defining equation for s(t) this random phase angle has been omitted for 
convenience of presentation. Using the result of Example 1.7 of Chapter 1 on a modulated 
random process, we may express the average power of the DSB-SC modulated signal 
component s(t) as C?A2P/2. With a noise spectral density of No/2, the average noise power 
in the message bandwidth W is equal to WNo. The channel signal-to-noise ratio of the 
DSB-SC modulation system is therefore 

C2A2P 
2WNo 


where the constant C? in the numerator ensures that this ratio is dimensionless. 

Next, we wish to determine the output signal-to-noise ratio of the system. Using the 
narrowband representation of the filtered noise 7(t), the total signal at the coherent detec- 
tor input may be expressed as 


s(t) + n(t) 
CA, cos(2mf,t)m(t) + 2;(t) cos(27f.t) — Q(t) sin(2af,t) 
where m,(t) and no(t) are the in-phase and quadrature components of m(z) with respect to 


the carrier. The output of the product-modulator component of the coherent detector is 
therefore 


(SNR)cpsa = (2.84) 


x(t) (2.85) 


u(t) = x(t) cos(27f,t) 
= 3CA gn(t) + 3n7(t) 
+ Z[CA ant) + m(t)] cos(4af.t) — inolt) sin(4zrf.t) 
The low-pass filter in the coherent detector in Figure 2.36 removes the high-frequency 
components of v(t), yielding the receiver output f 
y(t) = 7CAm(t) + m(t) (2.86) 
Equation (2.86) indicates the following: 


1. The message signal m(t) and in-phase noise component #;(¢) of the filtered noise n(z) 
appear additively at the receiver output. 
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2. The quadrature component 7o(¢) of the noise z(t) is completely rejected by the co- 
herent detector. 


These two results are independent of the input signal-to-noise ratio. Thus, coherent detec. 
tion distinguishes itself from other demodulation techniques in an important property: The 
output message component is unmutilated and the noise component always appears ad- 
ditively with the message, irrespective of the input signal-to-noise ratio. 

The message signal component at the receiver output is CA ,7(t)/2. Therefore, the 
average power of this component may be expressed as C*A2P/4, where P is the average 
power of the original message signal m(t) and C is the system-dependent scaling factor 
referred to earlier. 

In the case of DSB-SC modulation, the band-pass filter in Figure 2.36 has a band- 
width By equal to 2W in order to accommodate the upper and lower sidebands of the 
modulated signal s(t). It follows therefore that the average power of the filtered noise n(t) 
is 2WN,. From the discussion of narrowband noise presented in Section 1.11, we know 
that the average power of the (low-pass) in-phase noise component 7,(t) is the same as 
that of the (band-pass) filtered noise n(t). Since from Equation (2.86) the noise component 
at the receiver output is n;(t)/2, it follows that the average power of the noise at the receiver 
output is 


(5)°2WNo = 3WNo 
The output signal-to-noise for a DSB-SC receiver using coherent detection is therefore 


C*A2P/4 
(SNR)o, psesc = “WN,/2- 








2,87 
x C?AŻP í ) 
2WNo 
Using Equations (2.84) and (2.87), we obtain the figure of merit 
(SNR)o =1 (2.88) 


(SNR)c DSB-SC i 


Note that the factor C? is common to both the output and channel signal-to-noise ratios, 
and therefore cancels out in evaluating the figure of merit. 

Following through the noise analysis of a coherent detector for SSB, we find that, 
despite the fundamental differences between it and the coherent detector for DSB-SC mod- 
ulation, the figure of merit is exactly the same for both of them; see Problem 2.49. 

The important conclusions to-be drawn from the discussions presented in this section 
and Problem 2.49 are two-fold: 


1. For the same average transmitted or modulated signal power and the same average 
noise power in the message bandwidth, a coherent SSB receiver wili have exactly the 
same output signal-to-noise ratio as a coherent DSB-SC receiver. 

2. In both cases, the noise performance of the receiver is exactly the same as that ob- 
tained by simply transmitting the message signal in the presence of the same channel 
noise. The only effect of the modulation process is to translate the message signal to 
a different frequency band to facilitate its transmission over a band-pass channel. 


Simply put, neither DSB-SC modulation nor SSB modulation offers the means for a trade- 
off between improved noise performance and increased channel bandwidth. This is a se 
rious problem when high quality of reception is a requirement. 
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2,12 Noise in AM Receivers 
Using Envelope Detection 


The next noise analysis we perform is for an amplitude modulation (AM) system using an 
envelope detector in the receiver, as shown in the model of Figure 2.37. In a full AM signal, 
both sidebands and the carrier wave are transmitted, as shown by 


s(t) = A[1 + k,m(t)] cos(27f.t) (2.89) 


where A, cos(27f,t) is the carrier wave, m(t) is the message signal, and k, is a constant 
that determines the percentage modulation. In the expression for the amplitude-modulated 
signal component s(t) given in Equation (2.89), we see no need for the use of a scaling 
factor, because it is reasonable to assume that the carrier amplitude A, has the same units 
as the additive noise component. 

The average power of the carrier component in the AM signal s(t) is A2/2. The 
average power of the information-bearing component A kam(t) cos(27f.t) is A2k3P/2, 
where P is the average power of the message signal 7(t). The average power of the full 
AM signal s(t) is therefore equal to A2(1 + &2P)/2. As for the DSB-SC system, the average 
power of noise in the message bandwidth is WNo. The channel signal-to-noise ratio for 
AM is therefore 


A2(1 + k2P) 


(SNR) cam = 2WN, 


(2.90) 

To evaluate the output signal-to-noise ratio, we first represent the filtered noise n(t) 
in terms of its in-phase and quadrature components. We may therefore define the filtered 
signal x(t) applied to the envelope detector in the receiver model of Figure 2.37 as follows: 


x(t) = s(t) + n(t) 


[A. + A.kam(t) + n;(t)] cos(2af.t) — a(t) sin(27f,t) (2.91) 


It is informative to represent the components that comprise the signal x(t) by means of 
phasors, as in Figure 2.384. From this phasor diagram, the receiver output is readily ob- 
tained as 


envelope of x(t) 
{[A. + Akam(t) + mt)? + noth”? 


vt) (2.92) 


i 


The signal y(t) defines the output of an ideal envelope detector. The phase of x(t) is of no 
interest to us, because an ideal envelope detector is totally insensitive to variations in the 
phase of x(t). 

The expression defining y(t) is somewhat complex and needs to be simplified in some 
manner to permit the derivation of insightful results. Specifically, we would like to ap- 
proximate the output y(f) as the sum of a message term plus a term due to noise. In general, 
this is quite difficult to achieve. However, when the average carrier power is large com- 
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Figure 2.37 Model of AM receiver. 
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FIGURE 2.38 (a) Phasor diagram for AM wave plus narrowband noise for the case of high car- 
rier-to-noise ratio. (b) Phasor diagram for AM wave plus narrowband noise for the case of low 
carrier-to-noise ratio. 


pared with the average noise power, so that the receiver is operating satisfactorily, then 
the signal term A,[1 + kam(t)] will be large compared with the noise terms #;(t) and no(t), 
at least most of the time. Then we may approximate the output y(t) as (see Problem 2.51): 


y(t) = A, + AR m(t) + ni(t) (2.93) 


The presence of the DC or constant term A, in the envelope detector output y(t) of 
Equation (2.93) is due to demodulation of the transmitted carrier wave. We may ignore 
this term, however, because it bears no relation whatsoever to the message signal m(t). In 
any case, it may be removed simply by means of a blocking capacitor. Thus if we neglect 
the DC term A, in Equation (2.93), we find that the remainder has, except for scaling 
factors, a form similar to the output of a DSB-SC receiver using coherent detection. Ac- 
cordingly, the output signal-to-noise ratio of an AM receiver using an envelope detector 
is approximately 


Azk2P 


(SNR)o,am = 2WN, (2.94) 


Equation (2.94) is, however, valid only if the following two conditions are satisfied: 


1. The average noise power is small compared to the average carrier power at the 
envelope detector input. 

2. The amplitude sensitivity k, is adjusted for a percentage modulation less than or 
equal to 100 percent. 


Using Equations (2.90) and (2.94), we obtain the following figure of merit for amplitude 
modulation: 


(SNR)o| kP 


(SNR)c| ay 1 + A2P Ge) 





Thus, whereas the figure of merit of a DSB-SC receiver or that of an SSB receiver using 
coherent detection is always unity, the corresponding figure of merit of an AM receiver 
using envelope detection is always less than unity. In other words, the noise performance 
of a full AM receiver is always inferior to that of a DSB-SC receiver. This is due to the 
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wastage of transmitter power, which results from transmitting the carrier as a component 
of the AM wave. - 


> EXAMPLE 2.4 Single-Tone Modulation 


Consider the special case of a sinusoidal wave of frequency fẹ, and amplitude A,, as the 
modulating wave, as shown by . 


m(t) = A,, cos(2f,,t) 
The corresponding AM wave is 
s(t) = A[1 + u cos(2rf,,t)] cos(2af.t) 


where u = &,A,, is the modulation factor. The average power of the modulating wave m(t) is 
(assuming a load resistor of 1 ohm) 


P = 442, 


Therefore, using Equation (2.95), we get 








1 oa 
(SNR)o| _ 24h 
(SNR)c] am 1,242 
1 +g kaa (2.96) 
R 
2+ g? 


When u = 1, which corresponds to 100 percent modulation, we get a figure of merit equal 
to 1/3. This means that, other factors being equal, an AM system {using envelope detection) 
must transmit three times as much average power as a suppressed-carrier system (using co- 
herent detection) to achieve the same quality of noise performance. 


m THRESHOLD EFFECT 


When the carrier-to-noise ratio is small compared with unity, the noise term dominates 
and the performance of the envelope detector changes completely from that just described. 
In this case it is more convenient to represent the narrowband noise n(t) in terms of its 
envelope r(t) and phase y(t), as shown by 


n(t) = r(t) cos[2mft + w(t)] (2.97) 


The corresponding phasor diagram for the detector input x(t) = s(t) + a(t) is shown in 
Figure 2.385, where we have used the noise envelope as reference, because it is now the 
dominant term. To the noise phasor r(t) we have added a phasor representing the signal 
term A,[{1 + k,(¢)], with the angle between them being equal to the phase y(t) of the 
noise v(t). In Figure 2.386 it is assumed that the carrier-to-noise ratio is so low that the 
carrier amplitude A, is small compared with the noise envelope r(t), at least most of the 
time. Then we may neglect the quadrature component of the signal with respect to the 
noise, and thus find from Figure 2.386 that the envelope detector output is 


y(t) = r(t) + A. cos[y(t)] + Akamit) cos[w(t)] (2.98) 


This relation reveals that when the carrier-to-noise ratio is low, the detector output has 
no component strictly proportional to the message signal m(t). The last term of the ex- 
pression defining y(t) contains the message signal z(t) multiplied by noise in the form of 
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cos[i(t)]. From Section 1.11 we recall that the phase p(t) of the narrowband noise x(t) is 
uniformly distributed over 27 radians. It follows therefore that we have a complete loss 
of information in that the detector output does not contain the message signal m(t) at all. 
The loss of a message in an envelope detector that operates at a low carrier-to-noise ratio 
is referred to as the threshold effect. By threshold we mean a value of the carrier-to-noise 
ratio below which the noise performance of a detector deteriorates much more rapidly 
than proportionately to the carrier-to-noise ratio. It is important to recognize that every 
nonlinear detector (e.g., envelope detector) exhibits a threshold effect. On the other hand, 
such an effect does not arise in a coherent detector. 

A rigorous mathematical analysis of the threshold effect for the general case of an 
AM wave is beyond the scope of this book. In the next subsection we simplify matters by 
considering the case of an unmodulated carrier. Despite this simplification, we can still 
develop a great deal of insight into the threshold effect experienced in an envelope detector, 


General Formula for (SNR) in Envelope Detection* 
Consider an envelope detector whose input signal is defined by 
x(t) = A, cos(27f,t) + n(t) (2,99) 


where A, cos(27f,t) is the unmodulated carrier and x(t) is the sample function of band- 
limited, zero-mean, white Gaussian noise N(t). The power spectral density of N(t) is 


No 
Sed forlf- “ls (2.100) 
0 otherwise 


Representing the narrowband noise 7(t) in terms of its in-phase component 1,(t) and 
quadrature component no(t), we may express the noisy signal at the detector input as 


x(t) = (A, + m;(t)) cos(2mf,t) — ngolt) sin(27f.t) (2.101) 


The noise components 7;(t) and no(t) are zero-mean, jointly Gaussian, mutually indepen- 
dent low-pass random processes with identical power spectral densities (see Equation 
1.101): 


Snif = f) + SAf +f) for| fl = W 


2.102 
0 otherwise ( 


Sul) = Syolf) = { 


For the problem at hand, the input signal consists of an unmodulated carrier with 
average power equal to A2/2. The average noise power at the detector input is 








oy = 2WNo (2,103) 
The carrier-to-noiser ratio is therefore defined by 
Azs/2 
p= oy 
re (2.104) 
T AWN 


We may think of p as an input signal-to-noise ratio for the problem described herein. 


*A reader who is not interested in the mathematical details of how noise affects the envelope detection of an 
AM signal may skip che material up to Eq. (2.124) and read the two limiting cases of the formula in that equation. 
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However, determination of the output signal-to-noise ratio is a more difficult un- 
dertaking because the envelope detector output 


y(t) = Via, ¥ ml)? + nD) (2.105) 


is a nonlinear combination of signal and noise terms. With no clear-cut separation between 
signal and noise at the detector output y(t), how then do we isolate the contribution of 
the signal s(t) to y(t) from the contribution due to the noise n(t)? To resolve this issue, we 
adopt a heuristic approach based on signal averaging. Specifically, we introduce the fol- 
lowing two definitions: ` 


1. The mean output signal, s,, is the difference between the expectation of y(t) in the 
combined presence of signal and noise and the expectation of y(t) in the presence of 
noise alone, as shown by 


So = Ely(t)] — Ely.(e)] (2.106) 
where y(t) is itself defined by Equation (2.105) and y,(t) is defined by 
Yolt) = Vnt) + nalt) (2.107) 


2. The mean output noise power is the difference between the mean-square value of the 
detector output y(t) and the square of the mean value of y(t), as shown by 


var[y(¢)] = Ely?(t)] — (Ely(t)])? (2.108) 


On this basis, we define the output signal-to-noise ratio as 
2 


Fail (t)] 


From Section 1.12, we recall that the envelope detector output due to noise alone is 
Rayleigh distributed; that i is 


y a 
fuly) = 40% > ep- 7). yan (2.110) 


0 otherwise 





(SNR)o = (2.109) 


The expectation of y,(t) is therefore 
Ely.(t)] = l> yfr,(Y) dy 
o 2 2 
— y NAA 
J one ( 2 =) dy 


From the definition of the gamma function for real positive values of the argument x, we 
have 


(2.111) 





T(x) = f z7! exp(—z) dz (2.112) 
We may therefore rewrite Equation (2.111) as 


Epy,(t)] = Viewr(3} 


T 
n 


(2.113) 


140 


CHAPTER 2 & CONTINUOUS-WAVE MODULATION 


where we have used the value I'(3/2) = V7/2. To calculate the mean signal s, at the 
detector output, we also need the expectation of y(t). Due to the combined presence of 
signal and noise, we recall from Section 1.13 that y(t) is Rician distributed, as shown by 


2 2. 
y Y + Ac) (Ay 
f) = 3 02, exo( 20%; Joo ee for y= 0 (2.114) 


0 otherwise 





where Io(+) is the modified Bessel function of the first kind of zero order (see Appendix 3), 


Hence, 
2 2 2 
y y? + Az AY 
E -Í = = I s 
[y(t)] Th exo( 203; ) ( a dy (2 115) 


0 








Putting A,y/o% = u and recognizing that p = A2/2o%,, we may recast this expectation in 
the form 


ad 2 

IN = 2 ws 
E[y(t)] Bp)? exp(—p) [ u exo( =) te du (2.116) 
The integral in Equation (2.116) can be written in a concise form by using confluent 
hypergeometric functions; see Appendix 4. In particular, using the integral representation 


iene T(ml2)( [m 1 
fa 1 exp(—b?u°)Io(u) du = yi (ia (Za) (2.117) 





with m = 3, T(m/2) = Vri2 and b? = 1/4p, we may express the expectation of y(t) in 
terms of the confluent hypergeometric function ;F,(3/2;1;p) as 


Ely] = E on expt-al( (3s1»)) (2.118 


We may further simplify matters by using the following identity: 
exp(—#)(1F,(a38;z)) = 1Fy(B — a5B;~u) {2.119 


and so finally express the expectation of y(t) in the concise form 


Eiye) = E ox(s8(~}s4-») (2.120 


Thus using Equations (2.113) and (2.120) in Equation (2.106) yields the mean output 


signal as 
so = 5 2 I =c; P = 1 (2 121) 
a 2 Ni 1*1 7? > f $ 


whose dependence on the standard deviation ow of the noise x(t) is testimony to the 
intermingling of signal and noise at the detector output. 
Following a similar procedure, we may express the mean-square value of the detector 


output y(t) as 
o 3 2 2 
5 y yt + Az Ay 
Ep he 5 exo( 20%; Jr ox dy (2.122) 


= 2oN(Fi(—151;—p)) 
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Hence using Equations (2.120) and (2.122) in Equation (2.108) yields the mean output 
noise power as : 


varly(e)] = CEEA 3 arf -5ss- )) ) (2.123) 


Finally, using Equations (2.121) and (2.123) in Equation (2.109) yields the output 
signal-to-noise ratio for the envelope detection problem at hand as 


(a(o) wd 1) 
1 2 
(4m s(-185-»)] = ((-$:- )) 


Equation (2.124) is the general formula for the output signal-to-noise of an envelope 
detector whose input consists of an unmodulated carrier and band-limited, white Gaussian 
noise. Two limiting cases of this general formula are of particular interest: 





(SNR)o = (2.124) 


1. Large carrier-to-noise ratio. For large p, we may use the following asymptotic for- 
mula (see Appendix 4) 


1B(-Fsts-») = af for p> œ (2.125) 
Moreover, the following identity 
1F,(—1;1;-p) = 1 + p (2.126) 


holds exactly for all p. Accordingly, the use of Equations (2.125) and (2.126) in 
Equation (2.124) yields the following approximate formula for the output signal-to- 
noise ratio: 


(SNR)o=p for p> (2.127) 


where we have ignored contributions due to p¥” and p° in the numerator of Equation 
(2.124) as being subdominant compared to p for large p. Equation (2.127) shows 
that for large carrier-to-noise pthe envelope detector behaves like a coherent detector, 
in that the output signal-to-noise ratio is proportional to the input signal-to-noise 
ratio. 

2. Small carrier-to-noise ratio. For small p, we have (see Appendix 4) 


a 

iF, (asc;—p) = 1 — =P for p > 0 (2.128) 

Hence, using this asymptotic formula, we may approximate the output signal-to- 
noise ratio for small p as 


~ T 
16 — 47 (2.129) 
= 0.91 p? for p> 0 


(SNR)o 


where, in the denominator, we have ignored contributions due to p and p? as being 
subdominant compared to p° for small p. Equation (2.129) shows that for a small 
carrier-to-noise ratio, the output signal-to-noise ratio of the envelope detector is pro- 
portional to the squared input signal-to-noise ratio. 
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FIGURE 2.39 Output signal-to-noise ratio of an envelope detector for varying carrier-to-noise 
ratio. 


The conclusions drawn from the two limiting cases considered herein are that an 
envelope detector favors strong signals and penalizes weak signals. The phenomenon of 
weak signals being penalized by the detector is referred to as weak signal suppression, 
which is a manifestation of the threshold effect. 

Using the formula of Equation (2.124), in Figure 2.39 we have plotted the ourput 
signal-to-noise ratio (SNR)o of the envelope detector versus the carrier-to-noise ratio p 
using tabulated values of confluent hypergeometric functions. This figure also includes the 
two asymptotes for large p and small p. From Figure 2.39 we see that the output signal- 
to-noise ratio deviates from a linear behavior around a carrier-to-noise ratio of 10 dB (i.e. 
p= 10). 


| 2.13 Noise in FM Receivers 





Finally, we turn our attention to the noise analysis of a frequency modulation (FM) system, 
for which we use the receiver model shown in Figure 2.40. As before, the noise 1v(t) is 
modeled as white Gaussian noise of zero mean and power spectral density No/2. The 
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FIGURE 2.40 Model of an FM receiver. 
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received FM signal s(t) has a carrier frequency f, and transmission bandwidth By, such 
that only a negligible amount of power lies outside the frequency band f, + By/2 for 
positive frequencies, and similarly for negative frequencies. 

As in the AM case, the band-pass filter has a midband frequency f, and bandwidth 
By and therefore passes the FM signal essentially without distortion. Ordinarily, Br is 
small compared with the midband frequency f., so that we may use the narrowband 
representation for m(t), the filtered version of channel noise w(t), in terms of its in-phase 
and quadrature components. 

In an FM system, the message signal is transmitted by variations of the instantaneous 
frequency of a sinusoidal carrier wave, and its amplitude is maintained constant. Therefore, 
any variations of the carrier amplitude at the receiver input must result from noise or 
interference. The amplitude limiter, following the band-pass filter in the receiver model of 
Figure 2.40, is used to remove amplitude variations by clipping the modulated wave at 
the filter output almost to the zero axis. The resulting rectangular wave is rounded off by 
another band-pass filter that is an integral part of the limiter, thereby suppressing har- 
monics of the carrier frequency. Thus, the filter output is again sinusoidal, with an am- 
plitude that is practically independent of the carrier amplitude at the receiver input. 

The discriminator in the model of Figure 2.40 consists of two components: 


1. A slope network or differentiator with a purely imaginary frequency response that 
varies linearly with frequency. It produces a hybrid-modulated wave in which both 
amplitude and frequency vary in accordance with the message signal. 

2. An envelope detector that recovers the amplitude variation and thus reproduces the 
message signal. 


The slope network and envelope detector are usually implemented as integral parts of a 
single physical unit. 

The postdetection filter, labeled “baseband low-pass filter” in Figure 2.40, has a 
bandwidth that is just large enough to accommodate the highest frequency component of 
the message signal. This filter removes the out-of-band components of the noise at the 
discriminator output and thereby keeps the effect of the output noise to a minimum. 

The filtered noise #(t) at the band-pass filter output in Figure 2.40 is defined in terms 
of its in-phase and quadrature components by 


nt) = n;(t) cos(2af.t) — no(t) sin(27f.t) 


Equivalently, we may express n(t) in terms of its envelope and phase as 


n(t) = r(t) cos[(2mf,t) + y(t)] (2.130) 
where the envelope is 
r(t) = [nz(t) + natt)” (2.131) 
and the phase is 
= tanl Zl) 
y(t) = tan [e] (2.132) 


The envelope r(t) is Rayleigh distributed, and the phase y(t) is uniformly distributed over 
27 radians (see Section 1.12). 
The incoming FM signal s(t) is defined by 


s(t) = A, cos 2 + amk, | m(t) ar (2.133) 
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where A, is the carrier amplitude, f. is the carrier frequency, k; is the frequency sensitivity, 
and m(t} is the message signal. Note that, as with the standard AM, in FM there is no 
need to introduce a scaling factor in the definition of the modulated signal s(t), since it jg 
reasonable to assume that its amplitude A, has the same units as the additive noise com. 
ponent n(t). To proceed, we define 


b(t) = 27k, fmin dr (2.134) 


We may thus express s(t) in the simple form 
s(t) = A, cos[2mf,t + $(£)] (2.135) 
The noisy signal at the band-pass filter output is therefore 


x(t) = s(t) + n(t) 


= A, cos[2af,t + (t)] + r(t) cos[2rf.t + p(t) (2.136) 


It is informative to represent x(t) by means of a phasor diagram, as in Figure 2.41. In this 
diagram we have used the signal term as reference. The phase 6(t) of the resultant phasor 
representing x(t) is obtained directly from Figure 2.41 as 


r(t) sinl y(t) — ¢(t)] } 
A. + r(t) cos[w(t) — p(t) 


The envelope of x(t) is of no interest to us, because any envelope variations at the band- 
pass filter output are removed by the limiter. . 

Our motivation is to determine the error in the instantaneous frequency of the carrier 
wave caused by the presence of the filtered noise n(t). With the discriminator assumed 
ideal, its output is proportional to 6’ (t)/27 where 6’ (t) is the derivative of 6(t) with respect 
to time. In view of the complexity of the expression defining 6(t), however, we need to 
make certain simplifying approximations, so that our analysis may yield useful results. - 

We assume that the carrier-to-noise ratio measured at the discriminator input is large 
compared with unity. Let R denote the random variable obtained by observing (at some 
fixed time) the envelope process with sample function r(t) [due to the noise #(t)]. Then, at 
least most of the time, the random variable R is small compared with the carrier amplitude 
A., and so the expression for the phase 6(¢) simplifies considerably as follows: 


r(t) 
A. 


Ot) = H(t) + ran of (2.137) 





sin[y(t) — $(2)] (2.138) 


alt) = P(t) + 


or, using the expression for ¢ (t) given in Equation (2.134), 


r(t) 
A, 


Resultant 3 
. a(t} — b (1) 


Ae win ac A 





Blt) = 2k; | mlr) dr + 2 sinjylt) — l (2.139) 





FIGURE 2.41 Phasor diagram for FM wave plus narrowband noise for the case of high carrier- 
to-noise ratio. 
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The discriminator output is therefore 


v(t) = i delt) 
2a dt (2.140) 
= kym(t) + nalt) 
where the noise term #4(t) is defined by 
nalo) = z F(t) sine) = (0) (2.141) 


We thus see that provided the carrier-to-noise ratio is high, the discriminator output v(t) 
consists of the original message signal m(t) multiplied by the constant factor k,, plus an 
additive noise component #,(#). Accordingly, we may use the output signal-to-noise ratio 
as previously defined to assess the quality of performance of the FM receiver. Before doing 
this, however, it is instructive to see if we can simplify the expression defining the noise 
nalt). 

From the phasor diagram of Figure 2.41, we note that the effect of variations in the 
phase y(t) of the narrowband noise appear referred to the signal term ¢ (t). We know that 
the phase y(t) is uniformly distributed over 277 radians. It would therefore be tempting to 
assume that the phase difference y(t) — #(t) is also uniformly distributed over 27 radians. 
If such an assumption were true, then the noise #4(t) at the discriminator output would 
be independent of the modulating signal and would depend only on the characteristics of 
the carrier and narrowband noise. Theoretical considerations show that this assumption 
is justified provided that the carrier-to-noise ratio is high.” Then we may simplify Equation 
(2.141) as: 


nalt) = =~ © trl) sinto) (2.142) 


However, from the defining equations for r(t) and y(t), we note that the quadrature com- 
ponent #o(t) of the filtered noise z(t) is 








no(t) = r(t) sin[y(t)] (2.143) 
Therefore, we may rewrite Equation (2.142) as 
_ 1 duolt) 
nalt) = TrA, dt (2.144) 


This means that the additive noise n,(t) appearing at the discriminator output is deter- 
mined effectively by the carrier amplitude A, and the quadrature component no(t) of the 
narrowband noise n(t). 

The output signal-to-noise ratio is defined as the ratio of the average output signal 
power to the average output noise power. From Equation (2.140), we see that the message 
component in the discriminator output, and therefore the low-pass filter output, is k m(t). 
Hence, the average output signal power is equal to k2P, where P is the average power of 
the message signal m(t). 

To determine the average output noise power, we note that the noise 7,(t) at the 
discriminator output is proportional to the time derivative of the quadrature noise com- 
ponent no(t). Since the differentiation of a function with respect to time corresponds to 
multiplication of its Fourier transform by j27f, it follows that we may obtain the noise 
process ng(t) by passing 79(t) through a linear filter with a frequency response equal to 


janf A if 
2m., A, 
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This means that the power spectral density Sy,{f) of the noise q(t) is related to the power 
spectral density Sno(f) of the quadrature noise component #o(t) as follows: 


f? 
Snl) = 743 Snol) (2.145) 


With the band-pass filter in the receiver model of Figure 2.40 having an ideal fre. 
quency response characterized by bandwidth Br and midband frequency f., it follows that 
the narrowband noise n(t) will have a power spectral density characteristic that is similarly 
shaped. This means that the quadrature component no(t) of the narrowband noise n(t} 
will have the ideal low-pass characteristic shown in Figure 2.42a. The corresponding power 
spectral density of the noise #,(t) is shown in Figure 2.425; that is, 





Nof? Br 
sinia lS (2.146) 
0, otherwise 


In the receiver model of Figure 2.40, the discriminator output is followed by a low-pass 
filter with a bandwidth equal to the message bandwidth W. For wideband FM, we usually 
find that W is smaller than Br/2, where Br is the transmission bandwidth of the FM 
signal. This means that the out-of-band components of noise #4(t) will be rejected. There- 
fore, the power spectral density Sw (f) of the noise #o(t) appearing at the receiver output 
is defined by 


Nof? 
seia (lS (2.147) 
0, otherwise 


as shown in Figure 2.42c. The average output noise power is determined by integrating 
the power spectral density Sy,(f) from -W to W. We thus get the following result: 


w 
Average power of output noise = re f? df 
2 J-w 
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FIGURE 2.42 Noise analysis of FM receiver. (a) Power spectral density of quadrature compo- 
nent s(t) of narrowband noise nit). (b) Power spectral density of noise #4(t) at the discriminator 
output. (c) Power spectral density of noise no(t) at the receiver output. 
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Note that the average output noise power is inversely proportional to the average carrier 
power A2/2. Accordingly, in an FM system, increasing the carrier power has a noise- 
quieting effect. 

Earlier we determined the average output signal power as k3P. Therefore, provided 
the carrier-to-noise ratio is high, we may divide this average output signal power by the 
average Output noise power of Equation (2.148) to obtain the output signal-to-noise ratio 
3ACGP 
2NoW? 
The average power in the modulated signal s(t) is A2/2, and the average noise power in 
the message bandwidth is WNo. Thus the channel signal-to-noise ratio is 

Az 
2WNo 
Dividing the output signal-to-noise ratio by the channel signal-to-noise ratio, we get the 
following figure of merit for frequency modulation: 


(SNR)o| _ 3&7P 


(SNR)o.rm = (2.149) 


(SNR)cEm = (2.150) 


(SNR)cle, W? 


From Section 2.7 we recall that the frequency deviation Af is proportional to the 
frequency sensitivity ky of the modulator. Also, by definition, the deviation ratio D is equal 
to the frequency deviation Af divided by the message bandwidth W. In other words, the 
deviation ratio D is proportional to the ratio k,P”7/W. It follows therefore from Equation 
(2.151) that the figure of merit of a wideband FM system is a quadratic function of the 
deviation ratio. Now, in wideband FM, the transmission bandwidth B+ is approximately 
proportional to the deviation ratio D. Accordingly, we may state that when the carrier- 
to-noise ratio is high, an increase in the transmission bandwidth By provides a correspond- 
ing quadratic increase in the output signal-to-noise ratio or figure of merit of the FM 
system.The important point to note from this statement is that, unlike amplitude modu- 
lation, the use of frequency modulation does provide a practical mechanism for the ex- 
change of increased transmission bandwidth for improved noise performance. 





(2.151) 


> EXAMPLE 2.5 Single-Tone Modulation 


Consider the case of a sinusoidal wave of frequency fy as the modulating signal, and assume 
a peak frequency deviation Af. The modulated FM signal is thus defined by 


s(t) = A, cos] 2a + ra sn ft 


Therefore, we may write 
t 
ak, f mirt) dr = - sin(27rf,,t) 


Differentiating both sides with respect to time and solving for m(t), we get 


mt) = Af cos(2 T fmt) 
Ry 
Hence, the average power of the message signal m(t), developed across a 1-ohm load, is 
_ (Af? 
p= 2? 
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Substituting this result into the formula for the output signal-to-noise ratio given in Equation 
(2.149), we get 


3A2(Af)* 

(SNR)oem = ANW 
E 3A2p7 
4N,W 


where B = Af/W is the modulation index. Using Equation (2.151) to evaluate the correspond. 


ing figure of merit, we get 
-i(@) 
m 2\W 


= 3 
=78 


(SNR)o 
(SNR)c 








(2.152) 


It is important to note that the modulation index 8 = Af/W is determined by the bandwidth 
W of the postdetection low-pass filter and is not related to the sinusoidal message frequency 
fms except insofar as this filter is usually chosen so as to pass the spectrum of the desired 
message; this is merely a matter of consistent design. For a specified system bandwidth W, the 
sinusoidal message frequency f may lie anywhere between 0 and W and would yield the same 
output signal-to-noise ratio. 

It is of particular interest to compare the noise performance of AM and FM systems, 
An insightful way of making this comparison is to consider the figures of merit of the two 
systems based on a sinusoidal modulating signal. For an AM system operating with a sinu- 
soidal modulating signal and 100 percent modulation, we have (from Example 2.4): 


(SNR)o| _1 
(SNR)clau 3 
Comparing this figure of merit with the corresponding result described in Equation (2.152) 


for an FM system, we see that the use of frequency modulation offers the possibility of im- 
proved noise performance over amplitude modulation when 


> 4 
that is, 
V2 
p> = 0471 


We may therefore consider 6 = 0.5 as defining roughly the transition between narrowband 
FM and wideband FM. This statement, based on noise considerations, further confirms 
a similar observation that was made in Section 2.7 when considering the bandwidth of 
FM waves. 


E CAPTURE EFFECT 


The inherent ability of an FM system to minimize the effects of unwanted signals (e.g 
noise, as just discussed) also applies to interference produced by another frequency: 
modulated signal whose frequency content is close to the carrier frequency of the desired 
FM wave. However, interference suppression in an FM receiver works well only when the 
interference is weaker than the desired FM input. When the interference is the stronge! 
one of the two, the receiver locks onto the stronger signal and thereby suppresses the 
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desired FM input. When they are of nearly equal strength, the receiver fluctuates back and 
forth between them. This phenomenon is known as the capture effect, which describes 
another distinctive characteristic of frequency modulation. 


a FM THRESHOLD EFFECT 


The formula of Equation (2.149), defining the output signal-to-noise ratio of an FM re- 
ceiver, is valid only if the carrier-to-noise ratio, measured at the discriminator input, is 
high compared with unity. It is found experimentally that as the input noise power is 
increased so that the carrier-to-noise ratio is decreased, the FM receiver breaks. At first, 
individual clicks are heard in the receiver output, and as the carrier-to-noise ratio decreases 
still further, the clicks rapidly merge into a crackling or sputtering sound. Near the break- 
ing point, Equation (2.149) begins to fail by predicting values of output signal-to-noise 
ratio larger than the actual ones. This phenomenon is known as the threshold effect.’° The 
threshold is defined as the minimum carrier-to-noise ratio yielding an FM improvement 
that is not significantly deteriorated from the value predicted by the usual signal-to-noise 
formula assuming a small noise power. 

For a qualitative discussion of the FM threshold effect, consider first the case when 
there is a no signal present, so that the carrier wave is unmodulated. Then the composite 
signal at the frequency discriminator input is 


x(t) = [A + n;(t)] cos(2af,t) — no(t) sin(27f,t) (2.153) 


where ;,(t) and no(t) are the in-phase and quadrature components of the narrowband 
noise x(t) with respect to the carrier wave. The phasor diagram of Figure 2.43 displays 
the phase relations between the various components of x(t) in Equation (2.153). As the 
amplitudes and phases of n; (t) and no(t) change with time in a random manner, the point 
P, [the tip of the phasor representing x(t)] wanders around the point P (the tip of the 
phasor representing the carrier), When the carrier-to-noise ratio is large, 2;(t) and ngolt) 
are usually much smaller than the carrier amplitude A,, and so the wandering point P, in 
Figure 2.43 spends most of its time near point P2. Thus the angle 6(t) is approximately 
nQ(t/A, to within a multiple of 27. When the carrier-to-noise ratio is low, on the other 
hand, the wandering point P, occasionally sweeps around the origin and 6(t) increases or 
decreases by 27 radians. Figure 2.44 illustrates how in a rough way the excursions in 6(t), 
depicted in Figure 2.444, produce impulselike components in 6’(t) = d6/dt. The discrim- 
inator output v(t) is equal to 6’(t)/27. These impulselike components have different heights 
depending on how close the wandering point P, comes to the origin O, but all have areas 
nearly equal to +27 radians, as illustrated in Figure 2.44b. When the signal shown in 
Figure 2.44 is passed through the postdetection low-pass filter, corresponding but wider 
impulselike components are excited in the receiver output and are heard as clicks. The 
clicks are produced only when 6(t) changes by +27 radians. 





FIGURE 2.43 Phasor diagram interpretation of Equation (2.153). 
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FIGURE 2.44 Illustrating impulselike components in 6’(1) = d6(t)/dt produced by changes of 27 
in 6(t); (a) and (b) are graphs of @(¢) and 0'(t), respectively. 


From the phasor diagram of Figure 2.43, we may deduce the conditions required for 
clicks to occur. A positive-going click occurs when the envelope r(t) and phase (2) of the 
narrowband noise n(t) satisfy the following conditions: 

r(t) > A, 
w(t) < 7 = pit) + dlt) 
dyt) 


a> 
dt 2 


These conditions ensure that the phase 6(¢) of the resultant phasor x(t) changes by 27 
radians in the time increment dt, during which the phase of the narrowband noise increases 
by incremental amount dy(t). Similarly, the conditions for a negative-going click to occur 
are as follows: 
r(t) > A. 
w(t) > — > we) + ave) 
dyli) 


=< 
dt 0 


These conditions ensure that 6(t) changes by —27 radians during the time increment dt. 
The carrier-to-noise ratio is defined by 
Az 


= 2.154) 
P = 2B,No ( 





As p is decreased, the average number of clicks per unit time increases. When this number 
becomes appreciably large, threshold is said to occur. 
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The output signal-to-noise ratio is calculated as follows: 


1. The output signal is taken as the receiver output measured in the absence of noise. 
The average output signal power is calculated assuming a sinusoidal modulation that 
produces a frequency deviation Af equal to B7/2, so that the carrier swings back 
and forth across the entire input frequency band. 


2. The average output noise power is calculated when there is no signal present; that 
is, the carrier is unmodulated, with no restriction imposed on the value of the carrier- 
to-noise ratio p. 


On this heuristic basis, theory"! yields Curve I of Figure 2.45 presenting a plot of the 
output signal-to-noise ratio versus the carrier-to-noise ratio when the ratio B7/2 W is equal 
to 5. This curve shows that the output signal-to-noise ratio deviates appreciably from a 
linear function of the carrier-to-noise ratio p when p is less than about 10 dB. Curve II of 
Figure 2.45 shows the effect of modulation on the output signal-to-noise ratio when the 
modulating signal (assumed sinusoidal) and the noise are present at the same time. The 
average output signal power pertaining to curve II may be taken to be effectively the same 
as for curve I. The average output noise power, however, is strongly dependent on the 
presence of the modulating signal, which accounts for the noticeable deviation of curve I 
from curve I. In particular, we find that as p decreases from infinity, the output signal-to- 
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FIGURE 2.45 Dependence of output signal-to-noise ratio on input carrier-to-noise ratio for FM 
reciever. In curve I, the average output noise power is calculated assuming an unmodulated car- 

rier. In curve II, the average output noise power is calculated assuming a sinusoidally modulated 
carrier. Both curves I and II are calculated from theory. 
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noise deviates appreciably from a linear function of p when p is about 11 dB. Also when 
the signal is present, the resulting modulation of the carrier tends to increase the average 
number of clicks per second. Experimentally, it is found that occasional clicks are hearg 
in the receiver output at a carrier-to-noise ratio of about 13 dB, which appears to be only 
slightly higher than what theory indicates. Also it is of interest to note that the increase in. 
the average number of clicks per second tends to cause the output signal-to-noise ratio to 
fall off somewhat more sharply just below the threshold level in the presence of 
modulation. 

From the foregoing discussion we may conclude that threshold effects in FM receivers 
may be avoided in most practical cases of interest if the carrier-to-noise ratio p is equal to 
or greater than 20 or, equivalently, 13 dB. Thus using Equation (2.154) we find that the 
loss of message at the discriminator output is negligible if 


Az 
C > 2 
2BrNo c 





or, equivalently, if the average transmitted power A?/2 satisfies the condition 


A2 
S$ = 20BrNo (2.155) 


To use this formula, we may proceed as follows: 


1. For a specified modulation index 8 and message bandwidth W, we determine the 
transmission bandwidth of the FM wave, Br, using the universal curve of Figure 2.26 
or Carson’s rule. 

2. For a specified average noise power per unit bandwidth, No, we use Equation (2.155), 
to determine the minimum value of the average transmitted power A7/2 that is nec- 
essary to operate above threshold. 


a FM THRESHOLD REDUCTION 


In communication systems using frequency modulation, there is particular interest in re- 
ducing the noise threshold in an FM receiver so as to satisfactorily operate the receiver 
with the minimum signal power possible. Threshold reduction in FM receivers may be 
achieved by using an FM demodulator with negative feedback’? (commonly referred to as 
an FMEB demodulator), or by using a phase-locked loop demodulator. Such devices are 
referred to as extended-threshold demodulators, the idea of which is illustrated in Figure 
2.46, The threshold extension shown in this figure is measured with respect to the standard 
frequency discriminator (i.e., one without feedback). 

The block diagram of an FMFB demodulator"? is shown in Figure 2.47. We see that 
the local oscillator of the conventional FM receiver has been replaced by a voltage 
controlled oscillator (VCO) whose instantaneous output frequency is controlled by the 
demodulated signal. In order to understand the operation of this receiver, suppose for the 
moment that the VCO is removed from the circuit and the feedback loop is left open. 
Assume that a wideband FM signal is applied to the receiver input, and a second FM 
signal, from the same source but whose modulation index is a fraction smaller, is applied 
to the VCO terminal of the mixer. The output of the mixer would consist of the difference 
frequency component, because the sum frequency component is removed by the band-pass 
filter. The frequency deviation of the mixer output would be small, although the frequency 
deviation of both input FM waves is large, since the difference between their instantaneous 
deviations is small. Hence, the modulation indices would subtract and the resulting F 
wave at the mixer output would have a smaller modulation index. The FM wave wit 
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FIGURE 2.46 FM threshold extension. 


reduced modulation index may be passed through a band-pass filter, whose bandwidth 
need only be a fraction of that required for either wideband FM, and then frequency 
demodulated. It is now apparent that the second wideband FM signal applied to the mixer 
may be obtained by feeding the output of the frequency discriminator back to the VCO. 

It will now be argued that the signal-to-noise ratio of an FMFB receiver is the same 
as that of a conventional FM receiver with the same input signal and noise power if the 
carrier-to-noise ratio is sufficiently large. Assume for the moment that there is no feed- 
back around the demodulator. In the combined presence of an unmodulated carrier 
A, cos(27f,t) and narrowband noise 


nt) = nz(t) cos(2rf,t) — no(t) sin(27f,t) 


the phase of the composite signal x(z) at the limiter-discriminator input is approximately 
equal to #Q(t)/A,, assuming that the carrier-to-noise ratio is high. The envelope of x(t) is 
of no interest to us, because the limiter removes all variations in the envelope. Thus the 
composite signal at the frequency discriminator input consists of a small index phase- 
modulated wave with the modulation derived from the component no(t) of noise that is 
in phase quadrature with the carrier. When feedback is applied, the VCO generates a 
frequency-modulated signal that reduces the phase-modulation index of the wave in the 
band-pass filter output, that is, the quadrature component no(t) of noise. Thus we see that 
as long as the carrier-to-noise ratio is sufficiently large, the FMFB receiver does not respond 
to the in-phase noise component 2;(t), but that it would demodulate the quadrature noise 
component #o(t) in exactly the same fashion as it would demodulate signal modulation. 
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FIGURE 2.47 FM demodulator with negative feedback. 
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Signal and quadrature noise are reduced in the same proportion by the applied feedback, 
with the result that the baseband signal-to-noise ratio is independent of feedback. For large 
carrier-to-noise ratios, the baseband signal-to-noise ratio of an FMFB receiver is then the 
same as that of a conventional FM receiver. 

The reason that an FMEB receiver is able to extend the threshold is that, unlike a 
conventional FM receiver, it uses a very important piece of a priori information, namely, 
that even though the carrier frequency of the incoming FM wave will usually have large 
frequency deviations, its rate of change will be at the baseband rate. An FMFB demodp. 
lator is essentially a tracking filter that can track only the slowly varying frequency of a 
wideband FM signal, and consequently it responds only to a narrowband of noise centered 
about the instantaneous carrier frequency. The bandwidth of noise to which the FMFBR 
receiver responds is precisely the band of noise that the VCO tracks. The end result is that 
an FMEB receiver is capable of realizing a threshold extension on the order of 5-7 dB, 
which represents a significant improvement in the design of minimum power FM systems, 

Like the FMFB demodulator, the phase-locked loop (discussed later in Section 2,14) 
is also a tracking filter and, as such, the noise bandwidth to which it responds is precisely 
the band of noise tracked by the VCO. Indeed, the phase-locked loop demodulator offers 
a threshold extension capability with a relatively simple circuit. Unfortunately, the amount 
of threshold extension is not predictable by any existing theory, and it depends on signal 
parameters. Roughly speaking, improvement by a few (on the order of 2 to 3) decibels is 
achieved in typical applications, which is not as good as an FMFB demodulator. l 


PRE-EMPHASIS AND DE-EMPHASIS IN FM 


Equation (2.147) shows that the power spectral density of the noise at the output of an 
FM receiver has a square-law dependence on the operating frequency; this is illustrated in 
Figure 2.48a. In Figure 2.48b, we have included the power spectral density of a typical 
message source; audio and video signals typically have spectra of this form. In particular, 
we see that the power spectral density of the message usually falls off appreciably at higher 
frequencies. On the other hand, the power spectral density of the output noise increases 
rapidly with frequency. Thus around f = +W, the relative spectral density of the message 
is quite low, whereas that of the output noise is quite high in comparison. Clearly, the 
message is not using the frequency band allotted to it in an efficient manner. It may appear 
that one way of improving the noise performance of the system is to slightly reduce the 
bandwidth of the postdetection low-pass filter so as to reject a large amount of noise power 
while losing only a small amount of message power. Such an approach, however, is usually 
not satisfactory because the distortion of the message caused by the reduced filter band- 
width, even though slight, may not be tolerable. For example, in the case of music, we 
find that although the high-frequency notes contribute only a very small fraction of the 
total power, nonetheless, they contribute a great deal from an esthetic viewpoint. 

A more satisfactory approach to the efficient use of the allowed frequency band is 
based on the use of pre-emphasis in the transmitter and de-emphasis in the receiver, a 
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FIGURE 2.48 (a) Power spectral density of noise at FM receiver output, (b) Power spectral den 
sity of a typical message signal. 
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FIGURE 2.49 Use of pre-emphasis and de-emphasis in an FM system. 
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illustrated in Figure 2.49. In this method, we artificially emphasize the high-frequency 
components of the message signal prior to modulation in the transmitter, and therefore 
before the noise is introduced in the receiver. In effect, the low-frequency and high- 
frequency portions of the power spectral density of the message are equalized in such a 
way that the message fully occupies the frequency band allotted to it. Then, at the discrim- 
inator output in the receiver, we perform the inverse operation by de-emphasizing the 
high-frequency components, so as to restore the original signal-power distribution of the 
message. In such a process, the high-frequency components of the noise at the discriminator 
output are also reduced, thereby effectively increasing the output signal-to-noise ratio of 
the system. Such a pre-emphasis and de-emphasis process is widely used in commercial 
FM radio transmission and reception. 

In order to produce an undistorted version of the original message at the receiver 
output, the pre-emphasis filter in the transmitter and the de-emphasis filter in the receiver 
must ideally have frequency responses that are the inverse of each other. That is, if H,.(f) 
designates the frequency response of the pre-emphasis filter, then the frequency response 
Hyg(f) of the de-emphasis filter must ideally be (ignoring transmission delay) 


1 


H,.(f)’ 


This choice of frequency responses makes the average message power at the receiver output 
independent of the pre-emphasis and de-emphasis procedure. 

From our previous noise analysis in FM systems, assuming a high carrier-to-noise 
ratio, the power spectral density of the noise 74(¢) at the discriminator output is given by 
Equation (2.146). The modified power spectral density of the noise at the de-emphasis 
filter output is therefore 


Half) = “Wsfsw (2.156) 


Nof? z Br 
Half) 1°Sx,f) = 4 az Hell? UALS St (2.157) 
0, otherwise 


Recognizing, as before, that the postdetection low-pass filter has a bandwidth W that is, 
in general, less than By/2, we find that the average power of the modified noise at the 
receiver output is as follows: 


Average output noise No Í me 
== H, 2 2.15 
o with a A? /-w PIAA" af eee 


Because the average message power at the receiver output is ideally unaffected by the 
combined pre-emphasis and de-emphasis procedure, it follows that the improvement in 
output signal-to-noise ratio produced by the use of pre-emphasis in the transmitter and 
de-emphasis in the receiver is defined by 


average output noise power without pre-emphasis and de-emphasis 








: : 159 
average output noise power with pre-emphasis and de-emphasis 2 ) 
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Earlier we showed that the average output noise power without pre-emphasis and dẹ. 
emphasis is equal to (2N)W7/3A2); see Equation (2.148). Therefore, after cancellation of 
common terms, we may express the improvement factor I as 


2w? 
I= —y (2.160) 


3 |, PIH) af 





It must be emphasized that this improvement factor assumes the use of a high carrier-tp. 
noise ratio at the discriminator input in the receiver. 


œ EXAMPLE 2.6 


Asimple pre-emphasis filter that emphasizes high frequencies and is commonly used in practice 
is defined by the frequency-response 


-i+#f 
Hyelf) = 1 + 5 


which is closely realized by the RC-amplifier network shown in Figure 2.504, provided that 
R << r and 2afCr << 1 inside the frequency band of interest. The amplifier in Figure 2.504 
is intended to make up for the attenuation introduced by the RC network at low frequencies, 
The frequency parameter fo is 1/(22Cr). 

The corresponding de-emphasis filter in the receiver is defined by the frequency response 


1 
1 + fflfo 


which can be realized using the simple RC network of Figure 2.50b. 
The improvement in output signal-to-noise ratio of the FM receiver, resulting from the 
combined use of the pre-emphasis and de-emphasis filters of Figure 2.50, is therefore 


oOo 2Ww 
a” _fdf_ af 
3 af. wi + (fife (2.161) 


£ (Wifo)? 
3[(Wifo) — tan~*(W/fo)] 


Hac(f) = 





In commercial FM broadcasting, we typically have fọ = 2.1 kHz, and we may reason- 
ably assume W = 15 kHz. This set of values yields I = 22, which corresponds to an improve- 
ment of 13 dB in the output signal-to-noise ratio of the receiver. The output signal-to-noise 
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FIGURE 2.50 (a) Pre-emphasis filter. (b) De-emphasis filter. 


2.14 Computer Experiments: Phase-Locked Loop 157 


ratio of an FM receiver without pre-emphasis and de-emphasis is typically 40-50 dB. We see, 
therefore, that by using the simple pre-emphasis and de-emphasis filters shown in Figure 2.50, 
we can realize a significant improvement in the noise performance of the receiver. ad 


The use of the simple linear pre-emphasis and de-emphasis filters just described is an 
example of how the performance of an FM system may be improved by using the differ- 
ences between characteristics of signals and noise in the system. These simple filters also 
find application in audio tape-recording. Specifically, nonlinear pre-emphasis and de- 
emphasis techniques have been applied successfully to tape recording. These techniques! 
(known as Dolby-A, Dolby-B, and DBX systems) use a combination of filtering and dy- 
namic range compression to reduce the effects of noise, particularly when the signal level 
is low. 


2.14 Computer Experiments: 
| Phase-Locked Loop 


The experimental study presented in this section focuses on the use of a phase-locked loop 
for the demodulation of a frequency modulated signal. Before proceeding with the exper- 
iments, however, we first present a brief exposition of phase-locked loop theory. 

Basically, the phase-locked loop consists of three major components: a multiplier, a 
loop filter, and a voltage-controlled oscillator (VCO) connected together in the form of a 
feedback system, as shown in Figure 2.51. The VCO is a sinusoidal generator whose 
frequency is determined by a voltage applied to it from an external source. In effect, any 
frequency modulator may serve as a VCO. We assume that initially we have adjusted the 
VCO so that when the control voltage is zero, two conditions are satisfied: 


1. The frequency of the VCO is precisely set at the unmodulated carrier frequency f.. 


2. The VCO output has a 90 degree phase-shift with respect to the unmodulated carrier 
wave, 


Suppose then that the input signal applied to the phase-locked loop is an FM signal defined 
by 


s(t) = A, sin[2af.t + $4(t)] 
where A, is the carrier amplitude. With a modulating signal m(t), the angle #,(t) is related 


to m(t) by the integral 


y(t) = Ink; i m(t) dr 


Error 
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FIGURE 2.51 Phase-locked loop. 
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where k, is the frequency sensitivity of the frequency modulator. Let the VCO output ip 
the phase-locked loop be defined by 


r(t) = A, cos|27f.t + $2(E)] 


where A, is the amplitude. With a control voltage v(t) applied to the VCO input, the angle 
2(t) is related to v(t) by the integral 


halt) = 2rk, Í v(t) dt (2.162) 


where k, is the frequency sensitivity of the VCO, measured in Hertz per volt. The object 
of the phase-locked loop is to generate a VCO output 7(¢) that has the same phase angle 
(except for the fixed difference of 90 degrees) as the input FM signal s(t), The time-varying 
phase angle #,(t) characterizing s(t) may be due to modulation by a message signal m(t), 
in which case we wish to recover (t) and thereby produce an estimate of m(t). In other 
applications of the phase-locked loop, the time-varying phase angle ¢ ,(¢) of the incoming 
signal s(t) may be an unwanted phase shift caused by fluctuations in the communication 
channel; in this latter case, we wish to track ¢ (t) so as to produce a signal with the same 
phase angle for the purpose of coherent detection (synchronous demodulation). 


u MODEL OF THE PHASE-LOCKED Loop” 


To develop an understanding of the phase-locked loop, it is desirable to have a model of 
the loop. We start by developing a nonlinear model, which is subsequently linearized to 
simplify the analysis. According to Figure 2.51, the incoming FM signal s(t) and the VCO 
output r(t) are applied to the multiplier, producing two components: 


1. A high-frequency component, represented by the double-frequency term 
kmAA, sin[4mf.t + plt) + $2(t)] 
2. A low-frequency component represented by the difference-frequency term 
kmA-A, sin[oi(t) — p(t] 
where k, is the multiplier gain, measured in volt~?. 


The loop filter in the phase-locked loop is a low-pass filter, and its response to the high- 


_frequency component will be negligible. The VCO also contributes to the attenuation of 


this component. Therefore, discarding the high-frequency component (i.e., the double- 
frequency term), the input to the loop filter is reduced to 


elt) = kwAA, sinfol (2.163) 
where ¢,(t) is the phase error defined by 
h(t) = hilt) — bait) (2.164) 


t 


= p(t) — 2rk, 5 u(z) dr 


The loop filter operates on the error e(t) to produce an output v(t) defined by the convo- 
lution integral: 


co 


vít) = f e(t)h(t — 7) dr (2.165) 
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where b(t) is the impulse response of the loop filter. Using Equations (2.164) and (2.165) 
to relate #,(t) and @,(z), we obtain the following nonlinear integro-differential equation 
as the descriptor of the dynamic behavior of the phase-locked loop: 


dolt) dd, e 
seee) = det) — 27Ko fa sin[¢,(7)|b(t — 7) de (2.166) 


where Ko is as loop-gain parameter defined by 
Ko = kk AA, (2.167) 


The amplitudes A, and A, are both measured in volts, the multiplier gain k, in volt” * and 
the frequency sensitivity k, in Hertz per volt. Hence, it follows from Equation (2.167) that 
Ko has the dimensions of frequency. Equation (2.166) suggests the model shown in Figure 
2.52 for a phase-locked loop. In this model we have also included the relationship between 
u(t) and e(t) as represented by Equations (2.163) and (2.165). We see that the model of 
Figure 2.52 resembles the actual block diagram of Figure 2.51. The multiplier at the input 
of the phase-locked loop is replaced by a subtracter and a sinusoidal nonlinearity, and the 
VCO by an integrator. 

The sinusoidal nonlinearity in the model of Figure 2.52 complicates the task of an- 
alyzing the behavior of the phase-locked loop. It would be helpful to linearize this model 
to simplify the analysis and yet give a good approximate description of the loop’s behavior 
in certain modes of operation. When the phase error ¢,(2) is zero, the phase-locked loop 
is said to be in phase-lock. When ¢,(t) is at all times small compared with one radian, we 
may use the approximation 


sin[d(t)] = pelt) 


which is accurate to within 4 percent for #,(t) less than 0.5 radians. In this case, the loop 
is said to be near phase-lock, and the sinusoidal nonlinearity of Figure 2.52 may be dis- 
regarded. Under this condition, v(t) is approximately equal to m(t), except for the scaling 
factor k,/k,. 

The complexity of the phase-locked loop is determined by the frequency response 
H(f) of the loop filter. The simplest form of a phase-locked loop is obtained when 
H(f) = 1; that is, there is no loop filter, and the resulting phase-locked loop is referred to 
as a first-order phase-locked loop. A major limitation of a first-order phase-locked loop is 
that the loop gain parameter Ky controls both the loop bandwidth as well as the hold-in 
frequency range of the loop; the bold-in frequency range refers to the range of frequencies 


2nKy Drk, 
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FIGURE 2.52 Nonlinear model of the phase-locked loop. 
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for which the loop remains phase-locked to the input signal. We may overcome this lim: 
itation by using a loop filter with the frequency response 
ca: 
if 
where a is a constant. Then with this loop filter in place and the phase-locked loop oper. 


ating in its linear mode, we find from Equation (2.166) that the phase-locked loop behaves 
as a second-order feedback system, as shown by the standard frequency response 


Af) _ GUAN 
D) 1+2 Ofl + GMY 
where ®,(f) and ®,(f) are the Fourier transforms of ¢,(t) and (t), respectively. The 


system is parameterized by the natural frequency, fm and damping factor, £, which are 
respectively defined by 


H(f)=1+ (2.168) 








(2.169) 


fa = WaKo (2.170) 
and 
= [Ko 
b= iy l (2.171) 


The second-order phase-locked loop so described is the subject of the computer experi- 
ments presented next. 


Experiment 1: Acquisition Mode 


When a phase-locked loop is used for coherent detection (synchronous demodulation), the 
loop must first lock onto the input signal and then follow the variations of its phase angle 
with time. The process of bringing a loop into phase-lock is called acquisition, and the 
ensuing process of following angular variations in the input signal is called tracking. In 
the acquisition mode and quite possibly the tracking mode, the phase error ġ,(t) between 
the input signal s(t) and the VCO output r(t) will certainly be large, thereby mandating 
the use of the nonlinear model of Figure 2.52. However, a nonlinear analysis of the ac- 
quisition process based on this latter model is beyond the scope of this book. In this 
experiment, we use computer simulations to study the acquisition process and thereby 
develop insight into some of its features. 

Consider a second-order phase-locked loop using the loop filter of Equation (2.168) 
and having the following parameters: 


1 
Natural frequency f, = in Hz 


Damping factor ¢ = 0.3, 0.707, 1.0 


To accommodate variation in Z, the filter parameter a is varied in accordance with the 
formula 


fa 
2¢ 


which follows from Equations (2.170) and (2.171). Figure 2.53 presents the variation i1 
the phase error ¢,(t) with time for each of the three specified values of damping factor bs 


a= 
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FIGURE 2.53 Variation of the phase error for three different values of damping factor. 


assuming a frequency step of 0.125 Hz. These results show that the damping factor 
£ = 0.707 gives the best compromise between a fast response time and an underdamped 
oscillatory behavior. 


Experiment 2: Phase-Plane Portrait 


A phase-plane portrait is a family of trajectories, with each trajectory representing a single 
solution of Equation (2.166). For the second experiment we plot the phase-plane portrait 
of a second-order phase-locked loop for the case of sinusoidal modulation. The system 
parameters of the loop are as follows: 
_ 50 
Loop-gain parameter Ky = ae Hz 
50 
—= H 
2V2 


50 
Si idal modulation f. in = H 
inusoidal modulation frequency f, EV z 


Loop-natural frequency f,, = 





Figure 2.54 presents the phase-plane portrait of the phase-locked loop adjusted for 
critical damping, where the trajectories (frequency error versus phase error) are plotted 
for different starting points. From this portrait we make the following observations: 


1. For a sinusoidal nonlinearity, the phase-plane portrait is itself periodic with period 
2a in the phase error ¢,, but it is aperiodic in d,/dt. 
2. For an initial frequency error 
1 dd. 
K dt 
with an absolute value less than or equal to 1, the phase-locked loop is assured of 
attaining a stable (equilibrium) point at (0, 0) or (0, 277); the multiplicity of equilib- 
rium points is a manifestation of periodicity of the phase-plane portrait. 
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FIGURE 2.54 Phase-plane portrait for critical damping and sinusoidal modulation. 


3. For an initial frequency error 


1 dé. 
K dt 


with an absolute value equal to 2, we have a saddle point at (0, 1m) where the slightest 
perturbation applied to the phase-locked loop causes it to shift to the equilibrium 
point (0, 0) or (0, 277). 


| 2.15 Summary and Discussion 


In this chapter we studied the principles of continuous-wave (CW) modulation. This an- 
alog form of modulation uses a sinusoidal carrier whose amplitude or angle is varied in 
accordance with a message signal. We may thus distinguish two families of CW modula- 
tion: amplitude modulation and angle modulation. 





= AMPLITUDE MODULATION 


Amplitude modulation may itself be classified into four types, depending on the spectral 
content of the modulated signal. The four types of amplitude modulation and their prac- 
tical merits are as follows: 


1. Full amplitude modulation (AM), in which the upper and lower sidebands are trans- 
mitted in full, accompanied by the carrier wave. 

Accordingly, demodulation of an AM signal is accomplished rather simply in the receiver 
by using an envelope detector, for example. It is for this reason we find that full AM is 
commonly used in commercial AM radio broadcasting, which involves a single powerful 
transmitter and numerous receivers that are relatively inexpensive to build. 
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2. Double sideband-suppressed carrier (DSB-SC) modulation, in which only the upper 
and lower sidebands are transmitted. 

The suppression of the carrier wave means that DSB-SC modulation requires much less 
power than full AM to transmit the same message signal; this advantage of DSB-SC mod- 
ulation over full AM is, however, attained at the expense of increased receiver complexity. 
DSB-SC modulation is therefore well suited for point-to-point communication involving 
one transmitter and one receiver; in this form of communication, transmitted power is at 
a premium and the use of a complex receiver is therefore justifiable. 


3. Single sideband (SSB) modulation, in which only the upper sideband or lower sideband 
is transmitted. 

SSB modulation is the optimum form of CW modulation in the sense that it requires the 
minimum transmitted power and the minimum channel bandwidth for conveying a mes- 
sage signal from one point to another. However, its use is limited to message signals with 
an energy gap centered on zero frequency. 


4, Vestigial sideband modulation, in which almost all of one sideband and a vestige of 
the other sideband are transmitted in a prescribed complementary fashion. 

VSB modulation requires a channel bandwidth that is between that required for SSB and 
DSB-SC systems, and the saving in bandwidth can be significant if modulating signals with 
large bandwidths are being handled, as in the case of television signals and high-speed 
data. 


DSB-SC, SSB, and VSB are examples of linear modulation, whereas, strictly speaking, full 
AM is nonlinear. However, the deviation of full AM from linearity is of a mild sort. 
Accordingly, all four forms of amplitude modulation lend themselves readily to spectral 
analysis using the Fourier transform. 


ANGLE MODULATION 


Angle modulation ‘may be classified into frequency modulation (FM) and phase modula- 
tion (PM). In FM, the instantaneous frequency of a sinusoidal carrier is varied in propor- 
tion to the message signal. In PM, on the other hand, it is the phase of the carrier that is 
varied in proportion to the message signal. The instantaneous frequency is defined as the 
derivative of the phase with respect to time, except for the scaling factor 1/(27). Accord- 
ingly, FM and PM are closely related to each other; if we know the properties of the one, 
we can determine those of the other. For this reason, and because FM is commonly used 
in broadcasting, much of the material on angle modulation in the chapter was devoted 
to FM. 

Unlike amplitude modulation, FM is a nonlinear modulation process. Accordingly, 
spectral analysis of FM is more difficult than for AM. Nevertheless, by studying single- 
tone FM, we were able to develop a great deal of insight into the spectral properties of 
FM. In particular, we derived an empirical rule known as Carson’s rule for an approximate 
evaluation of the transmission bandwidth Bz of FM. According to this rule, Bris controlled 
by a single parameter: the modulation index £ for sinusoidal FM, or the deviation ratio 
D for nonsinusoidal FM. 


NOISE ANALYSIS 


We conclude the chapter on CW modulation systems by presenting a comparison of their 
noise performances. For this comparison, we assume that the modulation is produced by 
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FIGURE 2.55 Comparison of the noise performance of various CW modulation systems. Curve [; 
Full AM, u = 1. Curve II: DSB-SC, SSB. Curve III: FM, 8 = 2. Curve IV: FM, B = 5. (Curves 
IIL and IV include 13-dB pre-emphasis, de-emphasis improvement.) 


a sinusoidal wave. For the comparison to be meaningful, we also assume that the modu- 
lation systems operate with exactly the same channel signal-to-noise ratio. We may thus 
plot the output signal-to-noise ratio versus the channel signal-to-noise ratio as in Figure 
2.55 for the following modulation schemes: 


> Full AM with 100 percent modulation 
» Coherent DSB-SC, SSB 
> FM with B = 2 and B= 5 
Figure 2.55 also includes the AM and FM threshold effects. In making the comparison, it 


is informative to keep in mind the transmission bandwidth requirement of the modulation 
system in question. In this regard, we use a normalized transmission bandwidth defined by 


— Br 
wW 
where Bz is the transmission bandwidth of the modulated signal, and W is the message 


bandwidth. Table 2.4 presents the values of B, for the different CW modulation schemes. 
From Figure 2.55 and Table 2.4 we make the following observations: 


B, 


> Among the family of AM systems, SSB modulation is optimum with regard to noise 
performance as well as bandwidth conservation. i 

> The use of FM improves noise performance but at the expense of an excessive trans- 
mission bandwidth. This assumes that the FM system operates above threshold fot 
the noise improvement to be realized. 
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TABLE 2.4 Values of B,, for 
| various CW modulation schemes 


FM 
AM, DSB-SC SSB B=2 B=5 





2 1 8 16 





On an important point to conclude the discussion on CW modulation, only frequency 


modulation offers the capability to trade off transmission bandwidth for improved noise 
performance. The trade-off follows a square law, which is the best that we can do with 
CW modulation (i.e., analog communications). In Chapter 3 we describe pulse-code mod- 
ulation, which is basic to the transmission of analog information-bearing signals by a 
digital communication system, and which can indeed do much better. 


i NOTES AND REFERENCES 


1. 


It appears that the terms continuous wave and heterodyning were first used by Reginald 
Fessenden in the early 1900s. 


2. The Costas receiver is named in honor of its inventor; see the paper by Costas (1956). 


3. Bessel functions play an important role in the study of both analog and digital communi- 


10. 


11. 


12. 


cation systems. They can be of the so-called first kind or second kind. Appendix 3 discusses 
mathematical details and properties of both kinds of Bessel functions. A table of Bessel 
functions of the first kind is presented in Table A6.5. 


. Carson’s rule for the bandwidth of FM signals is named in honor of its originator; Carson 


and Fry (1937) wrote one of the early classic papers on frequency modulation theory. 


. The indirect method of generating a wideband FM wave was first proposed by Armstrong 


(1936). Armstrong was also the first to recognize the noise-robustness properties of fre- 
quency modulation. 


. Stereo multiplexing usually involves the use of frequency modulation for radio transmis- 


sion. However, it may also be transmitted using amplitude modulation as discussed in 
Problem 2.14; for more details, see the paper by Mennie (1978). 


. For detailed description of the superheterodyne receiver, see the Radio Engineering Hand- 


book edited by Henney (1958, pp. 19-34-19-41). 


. The qualitative study of threshold in envelope detection presented here follows Downing 


(1964, p. 71). 


. For a justification of the critical assumption on which the simplification presented in Equa- 


tion (2.142) rests, see Rice (1963). 


For a detailed discussion of the threshold effect in FM receivers, see the paper by Rice 
(1963) and the book by Schwartz, Bennett, and Stein (1966, pp. 129-163). 


Figure 2.45 is adapted from Rice (1963). The validity of the theoretical curve II in this 
figure has been confirmed experimentally; see Schwartz, Bennett, and Stein (1966, p. 153). 
For some earlier experimental work on the threshold phenomenon in FM, see the paper 
by Crosby (1937). 


The idea of using feedback around an FM demodulator was originally proposed by Chaffee 
(1939). 
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13. The treatment of the FMFB demodulator presented in Section 2.13 is based on the pape, 
by Enloe (1962); see also Roberts (1977, pp. 166-181). 


14. For a detailed discussion of Dolby systems mentioned in the latter part of Section 2.13, see 
Stremler (1990, pp. 732-734). 


15. For a full treatment of the nonlinear analysis of a phase-locked loop, see Gardner (1979) 
Lindsey (1972), and Viterbi (1966). 
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| PROBLEMS 


Amplitude Modulation 


2.1 Suppose that nonlinear devices are available for which the output current é, and input 
voltage y are related by 


i, = ayy; + azv? 
where a; and a; are constants. Explain how these devices may be used to provide: (a) a 


product modulator and (b) an amplitude modulator. 


2.2 Figure P2.2 shows the circuit diagram of a square-law modulator. The signal applied to 
the nonlinear device is relatively weak, such that it can be represented by a square law; 


v(t) = av (t) + a2v7(t) 


where a, and a, are constants, v(t) is the input voltage, and v(t) is the output voltage. 
The input voltage is defined by 


u(t) = A, cos(2af.t) + m(t) 


where s(t) is a message signal and A, cos(27f,t) is the carrier wave. 

(a) Evaluate the output voltage v(t). 

(b) Specify the frequency response that the tuned circuit in Figure P2.2 must satisfy in 
order to generate an AM signal with f, as the carrier frequency. 

(c) What is the amplitude sensitivity of this AM signal? 


Nonlinear 





Tuned tof. 


FIGURE P2.2 


2.3 Figure P2.3a shows the circuit diagram of a switching modulator. Assume that the carrie! 
wave c(t) applied to the diode is large in amplitude, so that the diode acts like an ideal 
switch: it presents zero impedance when forward biased (i.e., c(t) > 0). We may thus 
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approximate the transfer characteristic of the diode-load resistor combination by a piece- 
wise-linear characteristic defined as (see Figure P2.35) 


ae ae c(t) >0 
v= lo © aay<o 


That is, the load voltage vz(żt) varies periodically between the values v,(t) and zero at a 
rate equal to the carrier frequency f.. Hence, we may write 


u(t) = [A, cos(2mf.t) + m(t)]gr,(t) 
where g7,(t) is a periodic pulse train defined by 
(= 


n 





1.28 (-y™ 
alt= 5 += pA y q cosl2af.t(2n — 1) 


(a) Find the AM wave component contained in the output voltage v(t). 
(b) Specify the unwanted components in v(t) that need to be removed by a band-pass 
filter of suitable design. 


Up 
c(t) =A, cos (27f,0) 
© — 
mÒ © nO QE vÀ Slope = 1 
0 a 
(a) (b) 
FIGURE P2.3 


2.4 Consider the AM signal 


2.5 


s(t) = A [1 + u cos(2rf,,t)] cos(2 f-t) 


produced by a sinusoidal modulating signal of frequency f,,. Assume that the modulation 
factor is u = 2, and the carrier frequency f; is much greater than fn. The AM signal s(t) 
is applied to an ideal envelope detector, producing the output v(t). 
(a) Determine the Fourier series representation of u(t). 
(b) What is the ratio of second-harmonic amplitude to fundamental amplitude in v(t)? 
Figure P2.5 shows the circuit diagram of an envelope detector. It consists simply of a 
diode and resistor-capacitor (RC) filter. On a positive half-cycle of the input signal, the 
diode is forward-biased and the capacitor C charges up rapidly to the peak value of the 
input signal. When the input signal falls below this value, the diode becomes reverse- 
biased and the capacitor C discharges slowly through the load resistor R;. The discharging 
process continues until the next positive half-cycle. Thereafter, the charging-discharging 
routine is continued. 

(a) Specify the condition that must be satisfied by the capacitor C for it to charge rapidly 
and thereby follow the input voltage up to the positive peak when the diode is 
conducting. 

(b) Specify the condition which the load resistor R; must satisfy so that the capacitor C 
discharges slowly between positive peaks of the carrier wave, but not so long that the 
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capacitor voltage will not discharge at the maximum rate of change of the modulating 
wave. 


C R; Output 


AM wave 
w O 
FiGuReE P2.5 


2.6 Consider a square-law detector, using a nonlinear device whose transfer characteristic is 


defined by 
v(t) = awlt) + avilt) 


where a, and a, are constants, v(ż) is the input, and v(t) is the output. The input consists 
of the AM wave 


v(t) = A[1 + kam(t)] cos(27f.t) 


(a) Evaluate the output v(t). 
(b) Find the conditions for which the message signal m(t) may be recovered from v(t). 


2.7 The AM signal 
s(t) = A[1 + k,n(t)] cos(2af,2) 
is applied to the system shown in Figure P2.7. Assuming that | k,ym(t)| < 1 for all £ and 


the message signal m(t) is limited to the interval -W < f < W and that the cartier 
frequency f, > 2W show that m(t) can be obtained from the square-rooter output v;(Z), 


va Low-pass val Square 
s() => Squarer filter m footer vali) 


“ 2 
pil ra val) = Vol 


FIGURE P2.7 








2.8 Consider a message signal m(t) with the spectrum shown in Figure P2.8. The messag? 
bandwidth W = 1 kHz. This signal is applied to a product modulator, together with a 
carrier wave A, cos(2.7f,t), producing the DSB-SC modulated signal s(t). The modulated 
signal is next applied to a coherent detector. Assuming perfect synchronism between thé 
carrier waves in the modulator and detector, determine the spectrum of the detector 
output when: (a) the carrier frequency fe = 1.25 kHz and (b) the carrier frequency 
fa = 0.75 kHz. What is the lowest carrier frequency for which each component of the 
modulated signal s(t) is uniquely determined by m(t)? 
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MP) 
woow f 
FIGURE P2.8 


2.9 Figure P2.9 shows the circuit diagram of a balanced'modulator. The input applied to the 
top AM modulator is m(t), whereas that applied to the lower AM modulator is ~m(t); 
these two modulators have the same amplitude sensitivity. Show that the output s(t) of 
the balanced modulator consists of a DSB-SC modulated signal. 





AM 
modulator 





mit) 









ACOS (2af,1) 





Oscillator 











A,cos (2rrfot) 





AM 


~ml) modulator 


FIGURE P2.9 


2.10 A DSB-SC modulated signal is demodulated by applying it to a coherent detector. 


(a) Evaluate the effect of a frequency error Af in the local carrier frequency of the de- 
tector, measured with respect to the carrier frequency of the incoming DSB-SC signal. 

(b) For the case of a sinusoidal modulating wave, show that because of this frequency 
error, the demodulated signal exhibits beats at the error frequency. Illustrate your 
answer with a sketch of this demodulated signal. 


2.11 Consider the DSB-SC signal 
s(t) = A, cos(27f-t)m(t) 


where A, cos(27f,t) is the carrier wave and m(t) is the message signal. This modulated 
signal is applied to a square-law device characterized by 


y(t) = s*(t) 


The output y(t) is next applied to a narrowband filter with a passband magnitude response 
of one, midband frequency 2f., and bandwidth Af. Assume that Af is small enough to 
treat the spectrum of y(t) as essentially constant inside the passband of the filter. 


(a) Determine the spectrum of the square-law device output y(t). 
(b) Show that the filter output v(t) is approximately sinusoidal, given by 


Az 
v(t) = = E Af cos(47f.t) 


where E is the energy of the message signal z(t). 
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2.12 Consider the quadrature-carrier multiplex system of Figure 2.10. The multiplexed signa] 


2.13 


2.14 


2.15 


2.16 


s(t) produced at the transmitter output in Figure 2.104 is applied to a communication 
channel of frequency response H(f). The output of this channel is, in turn, applied to the 
receiver input in Figure 2.106. Prove that the condition 


Hf: + f) = E- fh Osf=W 


is necessary for recovery of the message signals m(t) and m(t) at the receiver outputs; 
f, is the carrier frequency, and W is the message bandwidth. Hint: Evaluate the spectra 
of the two receiver outputs. 


Suppose that in the receiver of the quadrature-carrier multiplex system of Figure 2.105 
the local carrier available for demodulation has a phase error with respect to the carrier 
source used in the transmitter. Assuming a distortionless communication channel between 
transmitter and receiver, show that this phase error will cause cross-talk to arise between 
the two demodulated signals at the receiver outputs. By cross-talk we mean that a portion 
of one message signal appears at the receiver output belonging to the other message signal, 
and vice versa. 


A particular version of AM stereo uses quadrature multiplexing. Specifically, the cartier 
A, cos(27f,t) is used to modulate the sum signal 


m,(t) = Vo + m(t) + m,{t) 


where Vo is a DC offset included for the purpose of transmitting the carrier component, 
m(t) is the left-hand audio signal, and m,(t) is the right-hand audio signal. The quadrature 
carrier A, sin(2-7f,t) is used to modulate the difference signal 


m(t) = m(t) — m,{(t) 


(a) Show that an envelope detector may be used to recover the sum m,(t) + m,(t) from 
the quadrature-multiplexed signal. How would you minimize the signal distortion. 
produced by the envelope detector? 

(b) Show that a coherent detector can recover the difference m(t) — m,(t). 

(c) How are the desired m(t) and m,(t) finally obtained? 

Using the message signal 

1 
mee 

determine and sketch the modulated waves for the following methods of modulation: 

(a) Amplitude modulation with 50 percent modulation. 

(b) Double sideband-suppressed carrier modulation. 

(c) Single sideband modulation with only the upper sideband transmitted. 

(d) Single sideband modulation with only the lower sideband transmitted. 


The Hilbert transform of a Fourier transformable signal m(t), denoted by 7#(¢), is defined 
by (see Appendix 2) 


Alt) = 1 Í ma) dt 
Tl-ot — T 
In the frequency domain, we have 


M(f) = -i sen(f)MIP) 
where m(t) = M(f), filt) = M(f), and sgn(f) is the signum function. 


2.17 
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Using the definition of the Hilbert transform, show that a single-sideband modu- 
lated signal resulting from the message signal s(t) and carrier cos(2 7f,t) of unit amplitude 
is given by (see Table 2.1) 


w(t) sin(2af.t) 


Nie 


s(t) = 5 mit) cos(27f.t) + 


where the minus sign corresponds to the transmission of the upper sideband and the plus 
sign corresponds to the transmission of the lower sideband. 


The local oscillator used for the demodulation of an SSB signal s(t) has a frequency error 
Af measured with respect to the carrier frequency f- used to generate s(t). Otherwise, 
there is perfect synchronism between this oscillator in the receiver and the oscillator sup- 
plying the carrier wave in the transmitter. Evaluate the demodulated signal for the fol- 
lowing two situations: 


(a) The SSB signal s(t) consists of the upper sideband only. 
(b) The SSB signal s(t) consists of the lower sideband only. 


Figure P2.18 shows the block diagram of Weaver’s method for generating SSB modulated 
waves. The message (modulating) signal 7(t) is limited to the band f = | f| = fp. The 
auxiliary carrier applied to the first pair of product modulators has a frequency fo, which 
lies at the center of this band, as shown by 


f= htt 


The low-pass filters in the in-phase and quadrature channels are identical, each with a 
cutoff frequency equal to (f, — f,)/2. The carrier applied to the second pair of product 
modulators has a frequency f, that is greater than (f, — f,)/2. Sketch the spectra at the 
various points in the modulator of Figure P2.18, and hence show that: 


(a) For the lower sideband, the contributions of the in-phase and quadrature channels 
are of opposite polarity, and by adding them at the modulator output, the lower 
sideband is suppressed. 

(b) For the upper sideband, the contributions of the in-phase and quadrature channels 
are of the same polarity, and by adding them, the upper sideband is transmitted. 

(c) How would you modify the modulator of Figure P2.18 so that only the lower side- 
band is transmitted? 


In-phase channel 
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Quadrature channel 


FIGURE P2.18 
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2.19 


2.20 


2.21 


2.22 


The spectrum of a voice signal m(t) is zero outside the interval fa = |f| = fe. To ensure 
communication privacy, this signal is applied to a scrambler that consists of the followin, 
cascade of components: product modulator, high-pass filter, second product modulator ` 
and low-pass filter. The carrier wave applied to the first product modulator has a fre. 
quency equal to f., whereas that applied to the second product modulator has a frequency 
equal to f, + f; both of them have unit amplitude. The high-pass and low-pass filters 
have the same cutoff frequency at f,. Assume that f, > fr- 


(a) Derive an expression for the scrambler output s(t), and sketch its spectrum. 


(b) Show that the original voice signal m(t) may be recovered from s(t) by using an 
unscrambler that is identical to the unit just described. 


In this problem we derive Equation (2.16) that defines the frequency response Ho(f) of 
the filter that operates on the message signal m(t) to produce m'(t) for VSB modulation, 
The signal m'(z), except for a scaling factor, constitutes the quadrature component of s(t), 
To do the derivation, we apply s(t) to the coherent detector of Figure P2.20 so as to 
recover a scaled version of the original message signal m(t). 


(a) Starting with the block diagram of Figure 2.12 for the generation of a VSB modulated 
wave, determine the Fourier transform V(f) of the product modulator output v(t) in 
Figure P2.20 in terms of the Fourier transform of the message signal m(t) and the 
frequency response H(f) of the band-pass filter in Figure 2.12. 

(b) Hence, by evaluating the Fourier transform of the low-pass filter output in Figure 
P2.20, determine the conditions that must be satisfied by Ho(f) in terms of H(f) to 
assure perfect recovery of the original message signal m(t), except for a scaling factor, 









vl) Output signal 
VSB modulated Product Low-pass |-> representing a scaled 
signal s( modulator filter version of the message 
signal m(t) 


A, COS (27,1) 
Figure P2.20 


The single-tone modulating signal m(t) = Am cos(27f,,t) is used to generate the VSB signal 
s(t) = È aAyA. cosl2afe + fadil +5 AmAd1 = a) cos[2mlf. ~ fat 


where a is a constant, less than unity, representing the attenuation of the upper side 

frequency. 

(a) Find the quadrature component of the VSB signal s(ż). 

(b) The VSB signal, plus the carrier A, cos(27f,t), is passed through an envelope detector. 
Determine the distortion produced by the quadrature component. 

(c) What is the value of constant a for which this distortion reaches its worst possible 
condition? $ 


In this problem we study the idea of mixing in a superheterodyne receiver. To be specific, 
consider the block diagram of the mixer shown in Figure P2.22 that consists of a product 
modulator with a local oscillator of variable frequency fi, followed by a band-pass filter. 
The input signal is an AM wave of bandwidth 10 kHz and carrier frequency that may lie 
anywhere in the range of 0.535 to 1.605 MHz; these parameters are typical of AM radio 
broadcasting. It is required to translate this signal to a frequency band centered at a fixed 
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intermediate frequency (IF) of 0.455 MHz. Find the range of tuning that must be provided 
in the local oscillator to achieve this requirement. 


vi) P 
st) Bane nase — valt) 


$ 





cos (2rf;t) ~ 
FIGURE P2.22 


2.23 Figure P2.23 shows the block diagram of a heterodyne spectrum analyzer. It consists of 
a variable-frequency oscillator, multiplier, band-pass filter, and root mean square (RMS) 
meter. The oscillator has an amplitude A and operates over the range fo to fo + W, where 
fo is the midband frequency of the filter and W is the signal bandwidth. Assume that 
fo = 2W, the filter bandwidth Af is small compared with fy and that the passband mag- 
nitude response of the filter is one. Determine the value of the RMS meter output for a 
low-pass input signal g(t). 
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frequency 
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FIGURE P2.23 


Angle Medulation 
2.24 Sketch the PM and FM waves produced by the sawtooth wave shown in Figure P2.24. 


mie) 





o To 2To 3To 


FIGURE P2.24 


2.25 In a frequency-modulated radar, the instantaneous frequency of the transmitted carrier 
is varied as in Figure P2.25, which is obtained by using a triangular modulating signal. 
The instantaneous frequency of the received echo signal is shown dashed in Figure P2.25, 
where ris the round-trip delay time. The transmitted and received echo signals are applied 
to a mixer, and the difference frequency component is retained. Assuming that for << 1, 
determine the number of beat cycles at the mixer output, averaged over one second, in 
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terms of the peak deviation Af of the carrier frequency, the delay 7, and the repetition 
frequency fo of the transmitted signal. 
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fe- Af 











Figure P2.25 


2.26 The instantaneous frequency of a sine wave is equal to f, — Af for |t| = T/2, and f, for 
|t| > T/2. Determine the spectrum of this frequency-modulated wave. Hint: Divide up 
the time interval of interest into three regions: —© < t < —T/2, —T/2 =< t = T/2, and 
T2 <t<, 

2.27 Single-sideband modulation may be viewed as a hybrid form of amplitude modulation 
and frequency modulation. Evaluate the envelope and instantaneous frequency of an SSB 
wave for the following two cases: 

(a) When only the upper sideband is transmitted. 

(b) When only the lower sideband is transmitted. 

2.28 Consider a narrowband FM signal approximately defined by 

s(t) = A, cos(27f.t) — BA, sin(2af,t) sin(27f,,t) 

(a) Determine the envelope of this modulated signal. What is the ratio of the maximum’ 
to the minimum value of this envelope? Plot this ratio versus £, assuming that £ is 
restricted to the interval 0 = 6 = 0.3. 

(b) Determine the average power of the narrowband FM signal, expressed as a percentage: 
of the average power of the unmodulated carrier wave. Plot this result versus £, 
assuming that £ is restricted to the interval 0 = £ = 0.3. 

(c) By expanding the angle 6;(t) of the narrow-band FM signal s(¢) in the form of a power 
series, and restricting the modulation index £ to a maximum value of 0.3 radians, 
show that 


O,(t) = Zaft + B sin(27f,,t) — £ sin?(27f,,t) 


What is the power ratio of third harmonic to fundamental component for $ = 0.3? 
2.29 The sinusoidal modulating wave 
mit) = Am cos(21ft) 
is applied to a phase modulator with phase sensitivity k,. The unmodulated carrier wave 
has frequency f, and amplitude A,. 
(a) Determine the spectrum of the resulting phase-modulated signal, assuming that the 
maximum phase deviation $, = &,A,, does not exceed 0.3 radians. 
(b) Construct a phasor diagram for this modulated signal, and compare it with that of 
the corresponding narrowband FM signal. 


2.30 Suppose that the phase-inodulated signal of Problem 2.29 has an arbitrary value for the 
maximum phase deviation £,. This modulated signal is applied to an ideal band-pass filter 
with midband frequency f. and a passband extending from f, — 1.5f,, to fe + 1.5 for 


2.31 


2.32 


2.33 


2.34 


2.35 
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Determine the envelope, phase, and instantaneous frequency of the modulated signal at 

the filter output as functions of time. 

A carrier wave is frequency-modulated using a sinusoidal signal of frequency fp, and 

amplitude A,,,. 

(a) Determine the values of the modulation index £ for which the carrier component of 
the FM signal is reduced to zero. For this calculation you may use the values of Jo(8) 
given in Table A6.5. 

(b) Ina certain experiment conducted with f,, = 1 kHz and increasing A, (starting from 
0 volts), it is found that the carrier component of the FM signal is reduced to zero 
for the first time when A,, = 2 volts. What is the frequency sensitivity of the modu- 
lator? What is the value of A,,, for which the carrier components is reduced to zero 
for the second time? 

An FM signal with modulation index £ = 1 is transmitted through an ideal band-pass 

filter with midband frequency f. and bandwidth 5f,,, where f, is the carrier frequency and 

fn is the frequency of the sinusoidal modulating wave. Determine the magnitude spectrum 
of the filter output. 

A carrier wave of frequency 100 MHz is frequency-modulated by a sinusoidal wave of 

amplitude 20 volts and frequency 100 kHz. The frequency sensitivity of the modulator 

is 25 kHz per volt. 

(a) Determine the approximate bandwidth of the FM signal, using Carson’s rule. 

(b) Determine the bandwidth by transmitting only those side frequencies whose ampli- 
tudes exceed 1 percent of the unmodulated carrier amplitude. Use the universal curve 
of Figure 2.26 for this calculation. 

(c) Repeat your calculations, assuming that the amplitude of the modulating signal is 

. doubled. 

(d) Repeat your calculations, assuming that the modulation frequency is doubled. 

Consider a wideband PM signal produced by a sinusoidal modulating wave 

Am cos(27rf,,t), using a modulator with a phase sensitivity equal to k, radians per volt. 

(a) Show that if the maximum phase deviation of the PM signal is large compared with 
one radian, the bandwidth of the PM signal varies linearly with the modulation fre- 
quency fin. 

(b) Compare this characteristic of a wideband PM signal with that of a wideband FM 
signal. 

Figure P2.35 shows the block diagram of a real-time spectrum analyzer working on the 

principle of frequency modulation. The given signal g(t) and a frequency-modulated signal 

s(t) are applied to a multiplier and the output g(t)s(t) is fed into a filter of impulse response 

h(t). The s(t) and h(t) are linear FM signals whose instantaneous frequencies vary linearly 

with time at opposite rates, as shown by 


s(t) = cos(2af,t — mkt?) 
A(t) = cos(2nf.t + mkt?) 


where k is a constant. Show that the envelope of the filter output is proportional to the 
magnitude spectrum of the input signal g(t) with kt playing the role of frequency f. Hint: 
Use the complex notations described in Appendix 2 for the analysis of band-pass signals 


and band-pass filters. 
gt) Filter — Output 


af) 
Figure P2.35 
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2.36 An EM signal with a frequency deviation of 10 kHz at a modulation frequency of $ kHz 
is applied to two frequency multipliers connected in cascade. The first multiplier doubles 
the frequency and the second multiplier triples the frequency. Determine the frequency 
deviation and the modulation index of the FM signal obtained at the second multiplier 
output. What is the frequency separation of the adjacent side frequencies of this FM 
signal? 

2.37 Figure P2.37 shows the block diagram of a wideband frequency modulator using the 
indirect method. This modulator is used to transmit audio signals containing frequencies 
in the range of 100 Hz to 15 kHz. The narrowband phase modulator is supplied with a 
carrier of frequency fı = 0.1 MHz by a crystal-controlled oscillator. A second crystal. 
controlled oscillator supplies a sinusoidal wave of frequency 9.5 MHz to the mixer. The 
system specifications are as follows: 

Carrier frequency at the transmitter output, fe = 100 MHz 

Minimum frequency deviation, Af = 75 kHz 

Maximum modulation index in the phase modulator = 0.3 radians 

(a) Calculate the frequency multiplication ratios 74, and m (preceding and following the 
mixer), which will satisfy these specifications. 

(b) Specify the values of the carrier frequency and frequency deviation at the various 
points in the modulator of Figure P2.37. 
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FIGURE P2.37 


2.38 Figure P2.38 shows the frequency-determining network of a voltage-controlled oscillator. 
Frequency modulation is produced by applying the modulating signa! A, sin(2 mft) plus 
a bias V, to a pair of varactor diodes connected across the parallel combination of a 
200-uH inductor and 100-pF capacitor. The capacitor of each varactor diode is related 
to the voltage V (in volts) applied across its electrodes by 


C = 100V~"? pF 


The unmodulated frequency of oscillation is 1 MHz. The VCO output is applied to a 
frequency multiplier to produce an FM signal with a carrier frequency of 64 MHz anda 
modulation index of 5. Determine (a) the magnitude of the bias voltage V, and (b) the 
amplitude A,, of the modulating wave, given that fẹ = 10 kHz. 


200 
BH 





Vp + Aq, Sin (Tfh 


Figure P2.38 
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2.39 The FM signal 


s(t) =A, cos] 2 + 2k; [ m(7) ar 


is applied to the system shown in Figure P2.39 consisting of a high-pass RC filter and an 
envelope detector. Assume that (a) the resistance R is small compared with the reactance 
of the capacitor C for all significant frequency components of s(t), and (b) the envelope 
detector does not load the filter. Determine the resulting signal at the envelope detector 
output, assuming that k,|m/(t)| < f; for all t. 






FM wave Envelope Output 
s(t) detector signal 


FIGURE P2.39 


2.40 Consider the frequency demodulation scheme shown in Figure P2.40 in which the incom- 
ing FM signal s(t) is passed through a delay line that produces a phase-shift of 7/2 radians 
at the carrier frequency f.. The delay-line output is subtracted from the incoming FM 
signal, and the resulting composite signal is then envelope-detected. This demodulator 
finds application in demodulating microwave FM signals. Assuming that 


s(t) = A, cos[2rf.t + B sin(27f,,t)] 


analyze the operation of this demodulator when the modulation index £ is less than unity 
and the delay T produced by the delay line is sufficiently small to justify making the 
approximations 


cos(27f,,T) = 1 
and 
sin(27f,,T) = Anfa T 


Envelope Output 
A detector signal 


FIGURE P2.40 
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2.41: Figure 2.41 shows the block diagram of a zero-crossing detector for demodulating an FM 
signal: It consists of a limiter, a pulse generator for producing a short pulse at each zero- 
crossing of the input, and a low-pass filter for extracting the modulating wave. 

(a) Show that the instantaneous frequency of the input FM signal is proportional to the 
number of zero crossings in the time interval t — (T,/2) to t + (T,/2), divided by T,. 
Assume that the modulating signal is essentially constant during this time interval. 
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(b) Ilustrate the operation of this demodulator, using the sawtooth wave of Figure P2234 
as the modulating wave. 


FM imi Pulse Low-pass Output 
wave > Limiter : generator > filter signal 


FIGURE P2.41 

















2.42 Suppose that the received signal in an FM system contains some residual amplitude mod. 
ulation of positive amplitude a(t), as shown by 


s(t) = a(t) cos[2af.t + $(£)] 


where f, is the carrier frequency. The phase ¢(t) is related to the modulating signal m(, 
by 


olt) = ak, | m(t) dr 


where k, is a constant. Assume that the signal s(t) is restricted to a frequency band of 
width Bz, centered at f., where By is the transmission bandwidth of the FM signal in the 
absence of amplitude modulation, and that the amplitude modulation is slowly varying 
compared with ¢(t). Show that the output of an ideal frequency discriminator produced 
by s(t) is proportional to a(t)m(t). Hint: Use the complex notation described in Appendix 
2 to represent the modulated wave s(t). 


2.43 (a) Let the modulated wave s(t) in Problem 2.42 be applied to a hard limiter, whose. 
output 2(t) is defined by E i 


z(t) = sgn[s(2)] 
_ f+, s(t) > 0 
= l-1, s(t) <0 
Show that the limiter output may be expressed in the form of a Fourier series as 


follows: 


a(t) = f È — cos[2af.t(2n + 1) + (2n + 1)d(¢)] 





(b) Suppose that the limiter output is applied to a band-pass filter with a passband mag: 
nitude response of one and bandwidth By centered about the carrier frequency fo 
where By is the transmission bandwidth of the FM signal in the absence of amplitude, 
modulation. Assuming that f. is much greater than By, show that the resulting filter 
output equals 


y(t) = Í cosl2nf.t + (è) 


By comparing this output with the original modulated signal s(t) defined in Problem 
2.42, comment on the practical usefulness of the result. 


2.44 (a) Consider an EM signal of carrier frequency f., which is produced by a modulating 
signal m(t). Assume that f, is large enough to justify treating this FM signal as 4 
narrowband signal. Find an approximate expression for its Hilbert transform. 

(b) For the special case of a sinusoidal modulating wave m(t) = A,,, cos(2 mft), find the 
exact expression for the Hilbert transform of the resulting FM signal. For this casê; 
what is the error in the approximation used in part (a)? 
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2.45 The single sideband version of angle modulation is defined by 
s(t) = exp[-d(¢)] cos[2af:t + o(2)] 
where (t) is the Hilbert transform of the phase function #(¢), and f, is the carrier 
frequency. 


(a) Show that the spectrum of the modulated signal s(t) contains no frequency compo- 
nents in the interval — f, < f < f., and is of infinite extent. ‘ 


(b) Given that the phase function 
olt) = B sin(27f,,t) 


where ß is the modulation index and f„ is the modulation frequency, derive the 
corresponding expression for the modulated wave s(t). 


Note: For Problems 2.44 and 2.45, you need to refer to Appendix 2 for a treatment of 
the Hilbert transform. 


Noise in CW Modulation Systems 


2.46 A DSB-SC modulated signal is transmitted over a noisy channel, with the power spectral 
density of the noise being as shown in Figure P2.46. The message bandwidth is 4 kHz 
and the carrier frequency is 200 kHz. Assuming that the average power of the modulated 
wave is 10 watts, determine the output signal-to-noise ratio of the receiver. 





-400 0 400 
FIGURE P2.46 


2.47 Evaluate the autocorrelation functions and cross-correlation functions of the in-phase and 
quadrature components of the narrowband noise at the coherent detector input for 
(a) the DSB-SC system, (b) an SSB system using the lower sideband, and (c) an SSB system 
using the upper sideband. 

2.48 In a receiver using coherent detection, the sinusoidal wave generated by the local oscillator 
suffers from a phase error 6(t) with respect to the carrier wave cos(27f,¢). Assuming that 
&t) is a sample function of a zero-mean Gaussian process of variance a, and that most 
of the time the maximum value of 6(z) is small compared with unity, find the mean-square 
error of the receiver output for DSB-SC modulation. The mean-square error is defined as 
the expected value of the squared difference between the receiver output and the message 
signal component of the receiver output. 

2.49 Following a procedure similar to that described in Section 2.11 for the DSB-SC receiver, 
extend this noise analysis to a SSB receiver. Specifically, evaluate the following: 
(a) The output signal-to-noise ratio. 
(b) The channel signal-to-noise ratio. 
Hence, show that the figure of merit for the SSB receiver is exactly the same as that for 
the DSB-SC receiver. Note that unlike the DSB-SC receiver, the midband frequency of the 
spectral density function of the narrowband-filtered noise at the front end of the SSB 
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receiver is offset from the carrier frequency f, by an amount equal to W/2, where W jg 
the message bandwidth. 
2.50 Let a message signal m(t) be transmitted using single-sideband modulation. The power 
spectral density of m(t) is 
„ifl ifs 
Sulf) = WwW’ 


0, otherwise 


where a and W are constants. White Gaussian noise of zero mean and power spectra} 
density No/2 is added to the SSB modulated wave at the receiver input. Find an expression 
for the output signal-to-noise ratio of the receiver. 

2.51 Consider the output of an envelope detector defined by Equation (2.92), which is repro. 
duced here for convenience 


y(t) = {[A, + Akam(t) + m(t)P + rol 
(a) Assume that the probability of the event 
|no(t)| > eA,|1 + kamlt)| 
is equal to or less than 5,, where e << 1. What is the probability that the effect of 
the quadrature component no(t) is negligible? 


(b) Suppose that k, is adjusted relative to the message signal m(t) such that the probability 
of the event 


A{1 + kam(t)] + m(t) < 0 
is equal to ô. What is the probability that the approximation 
y(t) = A[1 + kam(t)] + nt) 


is valid? 
(c) Comment on the significance of the result in part (b) for the case when ô, and ô, are. 
both small compared with unity. 

2.52 An unmodulated carrier of amplitude A, and frequency f, and band-limited white noise 
are summed and then passed through an ideal envelope detector. Assume the noise spec- 
tral density to be of height N,/2 and bandwidth 2 W, centered about the carrier frequency 
f- Determine the output signal-to-noise ratio for the case when the carrier-to-noise ratio 
is high. 

2.53 Let R denote the random variable obtained by observing the output of an envelope de- 
tector at some fixed time. Intuitively, the envelope detector is expected to be operating 
well into the threshold region if the probability that the random variable R exceeds the 
carrier amplitude A, is 0.5. On the other hand, if this same probability is only 0.01, the 
envelope detector is expected to be relatively free of loss of message and the threshold 
effect. 

(a) Assuming that the narrowband noise at the detector input is white, zero-mean, Gaus- 
sian with spectral density No/2 and the message bandwidth is W, show that the prob- 
ability of the event R = A, is 


P(R = A.) = exp(— p) 
where p is the carrier-to-noise ratio: 
Az 
~ 4WN, 


(b) Using the formula for this probability, calculate the carrier-to-noise ratio when (1) the 
envelope detector is expected to be well into the threshold region, and (2) it is expected 
to be operating satisfactorily. 





p 


2.54 


2.55 


2.56 


2.57 


2.58 
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Consider a phase modulation (PM) system, with the modulated wave defined by 
s(t) = A, cos[2af.t + k,m(t)] 


where k, is a constant and m(t) is the message signal. The additive noise x(t) at the phase 
detector input is 


n(t) = n,(t) cos(27f,t) — rolt) sin(2zf,t) 


Assuming that the carrier-to-noise ratio at the detector input is high compared with unity, 
determine (a) the output signal-to-noise ratio and (b) the figure of merit of the system. 
Compare your results with the FM system for the case of sinusoidal modulation. 

An FDM system uses single-sideband modulation to combine 12 independent voice signals 
and then uses frequency modulation to transmit the composite baseband signal. Each 
voice signal has an average power P and occupies the frequency band 0.3 to 3.4 kHz; the 
system allocates it a bandwidth of 4 kHz. For each voice signal, only the lower sideband 
is transmitted. The subcarrier waves used for the first stage of modulation are defined by 


cy(t) = A, cos(2mkfot), 0=k=11 
The received signal consists of the transmitted FM signal plus white Gaussian noise of 


zero mean and power spectral density Nj/2. 

(a) Sketch the power spectral density of the signal produced at the frequency discrimi- 
nator output, showing both the signal and noise components. 

(b) Find the relationship between the subcarrier amplitudes A, so that the modulated 
voice signals have equal signal-to-noise ratios. 

In the discussion on FM threshold effect presented in Section 2.13, we described the 

conditions for positive-going and negative-going clicks in terms of the envelope r(t) and 

phase y(t) of the narrowband noise x(t). Reformulate these conditions in terms of the in- 

phase component ,(t) and quadrature component no(t) of n(t). 

By using the pre-emphasis filter shown in Figure 2.504 and with a voice signal as the 

modulating wave, an FM transmitter produces a signal that is essentially frequency- 

modulated by the lower audio frequencies and phase-modulated by the higher audio 

frequencies. Explain the reasons for this phenomenon. 

Suppose that the transfer functions of the pre-emphasis and de-emphasis filters of an FM 

system are scaled as follows: 


Hyelf) = hı a if) 


and 


1 1 
nat = (Team) 


The scaling factor k is to be chosen so that the average power of the emphasized message 

signal is the same as that of the original message signal m(t). 

(a) Find the value of k that satisfies this requirement for the case when the power spectral 
density of the message signal m(t) is 


So 
Saf) = 71+ (F? 


0, elsewhere 


-W=<f=<W 


(b) What is the corresponding value of the improvement factor I produced by using this 
pair of pre-emphasis and de-emphasis filters? Compare this ratio with that obtained 
in Example 2.6. The improvement factor J is defined by Equation (2.160). 
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2.59 A phase modulation (PM) system uses a pair of pre-emphasis and de-emphasis filter, 
defined by the transfer functions 


Hpeiet 
fo 
and 
s 1 + (iflfo) 
Show that the improvement in output signal-to-noise ratio produced by using this pair of 
filters is 
Wifo 


oe ee 
tan7'(W/fo) 
where W is the message bandwidth. Evaluate this improvement for the case when 


W = 15 kHz and fy = 2.1 kHz, and compare your result with the corresponding value 
for an FM system. 


Computer Experiments 


2.60 In this experiment we study the behavior of the envelope detector shown in Figure P2.5 
for the following specifications: 
Source resistance, R, = 75 Q 
Load resistance, R; = 10 kQ 
Capacitance, C = 0.01 uF 
The diode has a resistance of 25 Q when it is forward-biased and infinite resistance when 
reverse-biased. 

Compute the waveform of the envelope detector output, assuming an. input sinu: 
soidal AM wave with 50 percent modulation. The modulation frequency is 1 kHz, and 
the carrier frequency is 20 kHz. 

2.61 In this experiment we continue the study of the phase-locked loop considered in Section 
2.14: 


(a) Compute variations in the instantaneous frequency of the voltage-controlled oscillator 
m the loop for the following loop parameters: 


50 
Loop-gain parameter, Ky = ae Hz 
Natural frequency, fy = z- Hz 
aaa A E 


Damping factor, ¢ = 0.707 
Perform the computations for the following values of frequency step: Af = 0.125, 


7 2 
0.5, 7p 5 Hz. 
(b) For the parameters of the phase-locked loop as specified in Experiment 2 in Section | 
2.14, compute how variations in the relative frequency deviation Af + fmf? affect 
the peak phase error of the phase-locked loop. 





PULSE MODULATION 


This chapter, representing the transition from analog to digital communications, covers 
the following topics: 


p Sampling, which is basic to all forms of pulse modulation. 
p Pulse-amplitude modulation, which is the simplest form of pulse modulation. 


p Quantization, which, when combined with sampling, permits the representation of an 
analog signal in discrete form in both amplitude and time. 


p Pulse-code modulation, which is the standard method for the transmission of an analog 
message signal by digital means. 


p Time-division multiplexing, which provides for the time sharing of a common channel by 
a plurality of users by means of pulse modulation. 


> Digital multiplexers, which combine many slow bit streams into a single faster stream. 


> Other forms of digital pulse modulation, namely, delta modulation and differential pulse- 
code modulation. 


P> Linear prediction, which is basic to the encoding of analog message signals at reduced bit 
rates as in differential pulse-code modulation. 


Š Adaptive forms of differential pulse-code modulation and delta modulation. 


> The MPEG-1/audio coding standard, which is a transparent, perceptually lossless 
compression system. 


j 3.1 Introduction 


In continuous-wave (CW) modulation, which we studied in Chapter 2, some parameter 
of a sinusoidal carrier wave is varied continuously in accordance with the message signal. 
This is in direct contrast to pulse modulation, which we study in the present chapter. In 
pulse modulation, some parameter of a pulse train is varied in accordance with the message 
signal. We may distinguish two families of pulse modulation: analog pulse modulation 
and digital pulse modulation. In analog pulse modulation, a periodic pulse train is used 
as the carrier wave, and some characteristic feature of each pulse (e.g., amplitude, duration, 
or position) is varied in a continuous manner in accordance with the corresponding sample 
value of the message signal. Thus in analog pulse modulation, information is transmitted 
basically in analog form, but the transmission takes place at discrete times. In digital pulse 
modulation, on the other hand, the message signal is represented in a form that is discrete 
in both time and amplitude, thereby permitting its transmission in digital form as a se- 
quence of coded pulses; this form of signal transmission has no CW counterpart. 
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The use of coded pulses for the transmission of analog information-bearing signals 
represents a basic ingredient in the application of digital communications. This chapter 
may therefore be viewed as a transition from analog to digital communications in oup 
study of the principles of communication systems. We begin the discussion by describing 
the sampling process, which is basic to all pulse modulation systems, whether they are 
analog or digital. 


| 3.2 Sampling Process 





The sampling process is usually described in the time domain. As such, it is an operation 
that is basic to digital signal processing and digital communications. Through use of the 
sampling process, an analog signal is converted into a corresponding sequence of samples 
that are usually spaced uniformly in time. Clearly, for such a procedure to have practical 
utility, it is necessary that we choose the sampling rate properly, so that the sequence of 
samples uniquely defines the original analog signal. This is the essence of the sampling 
theorem, which is derived in what follows. 

Consider an arbitrary signal g(t) of finite energy, which is specified for all time. A 
segment of the signal g(t) is shown in Figure 3.14. Suppose that we sample the signal g(t) 
instantaneously and at a uniform rate, once every T, seconds. Consequently, we obtain an 
infinite sequence of samples spaced T, seconds apart and denoted by {g(#T,)}, where n 
takes on all possible integer values. We refer to T, as the sampling period, and to its 
reciprocal f, = 1/T, as the sampling rate. This ideal form of sampling is called instantaneous 
sampling. 

Let g(t) denote the signal obtained by individually weighting the elements of a pe- 
riodic sequence of delta functions spaced T, seconds apart by the sequence of numbers 
{g(#T,)}, as shown by (see Figure 3.15) 


glt) = 2 g(nT,) 5(t — nT.) (3.4) 


We refer to g,(t) as the ideal sampled signal. The term 5(t — nT.) represents a delta function 
positioned at time t = nT,. From the definition of the delta function, we recall that such 
an idealized function has unit area; see Appendix 2. We may therefore view the multiplying 
factor g(nT,) in Equation (3.1) as a “mass” assigned to the delta function 6(t — #T,). A 
delta function weighted in this manner is closely approximated by a rectangular pulse of 


a) gs) 


, —_t 


ELEA | 
3 cae 


(a) (b) 





FIGURE 3.1 The sampling process. (a) Analog signal. (b) Instantaneously sampled version of the 
analog signal. 
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duration At and amplitude g(nT)/At; the smaller we make At the better will be the 
approximation. 

Using the table of Fourier-transform pairs, we may write (see the last item of Table 
A6.3) 


gilt) =f D GUY- mf.) (3.2) 
where G(f) is the Fourier transform of the original signal g(t), and f, is the sampling rate. 
Equation (3.2) states that the process of uniformly sampling a continuous-time signal of 
finite energy results in a periodic spectrum with a period equal to the sampling rate. 

Another useful expression for the Fourier transform of the ideal sampled signal g,(t) 
may be obtained by taking the Fourier transform of both sides of Equation (3.1) and noting 
that the Fourier transform of the delta function 5(t — nT,) is equal to exp(—j2anfT,). Let 
Gelf) denote the Fourier transform of g,(t). We may therefore write 


Gaf) = È gaT.) exp(—j2anfT,) (3.3) 
This relation is called the discrete-time Fourier transform. It may be viewed as a complex 
Fourier series representation of the periodic frequency function G,(f), with the sequence 
of samples {g(nT,)} defining the coefficients of the expansion. 

The relations, as derived here, apply to any continuous-time signal g(t) of finite 
energy and infinite duration. Suppose, however, that the signal g(t) is strictly band-limited, 
with no frequency components higher than W Hertz. That is, the Fourier transform G(f) 
of the signal g(t) has the property that G(f) is zero for | f| = W, as illustrated in Figure 
3.2a; the shape of the spectrum shown in this figure is intended for the purpose of illus- 
tration only. Suppose also that we choose the sampling period T, = 1/2W. Then the 
corresponding spectrum G,(f) of the sampled signal g,{t) is as shown in Figure 3.2. 
Putting T, = 1/2 W in Equation (3.3) yields 


oin= È a) (2) (3.4 


n=O 





From Equation (3.2), we readily see that the Fourier transform of g(t) may also be 
expressed as 


GA) = fC) + £ > GU- mf) (3.5) 


m=— o 
m+0 











-2f, r —W 0 Woof 2f 


(a) (b) 


FIGURE 3.2 (a) Spectrum of a strictly band-limited signal g(t): (b) Spectrum of the sampled 
version of g(t) for a sampling period T, = 1/2W. 
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Hence, under the following two conditions: 


1. Gf) = 0 for| f| = W 
2. f,=2W 
we find from Equation (3.5) that 


GIA =z Gaf -W<f<W 3.6 


Substituting Equation (3.4) into (3.6), we may also write 


1 < n jin. 
Gi) = 59 > (5) cxp( 2), -W<f<W (3.7) 
Therefore, if the sample values g(n/2W) of a signal g(t) are specified for all #, then the 
Fourier transform G(f) of the signal is uniquely determined by using the discrete-time 
Fourier transform of Equation (3.7). Because g(t) is related to G(f) by the inverse Fourier 
transform, it follows that the signal g(t) is itself uniquely determined by the sample values 
g(nl2W) for —œ < n< œ, In other words, the sequence {g(n/2W)} has all the information 
contained in g(t). 

Consider next the problem of reconstructing the signal g(t) from the sequence of 
sample values {g(n/2.W)}. Substituting Equation (3.7) in the formula for the inverse Fourier 
transform defining g(t) in terms of G(f), we get 


æ 


att) = | GIP) explj2afi) df 


Di J a n junf ? 
= _wiW Slow e| — Ty exp(j2aft) df 


=—o 


Interchanging the order of summation and integration: 


= n 1 (” , n : 
a(t) = 28 JW aly jamf\t — sy df (3.8): 


The integral term in Equation (3.8) is readily evaluated, yielding the final result 


_ Š n \ sin(2rWt — na) 
git) D, (5) (20Wt — nn) 


= Do e(z) sinc(2Wt — n), -œ < £< œ% 
Equation (3.9) provides an interpolation formula for reconstructing the original signal g(t) 
from the sequence of sample values {g(/2W)}, with the sinc function sinc(2 Wt) playing 
the role of an interpolation function. Each sample is multiplied by a delayed version of 
the interpolation function, and all the resulting waveforms are added to obtain g(t). 

We may now state the sampling theorem for strictly band-limited signals of finite 
energy in two equivalent parts, which apply to the transmitter and receiver of a pulse- 
modulation system, respectively: 





(3.9) 


1. A band-limited signal of finite energy, which has no frequency components higher 
than W Hertz, is completely described by specifying the values of the signal at instants 
of time separated by 1/2W seconds. 

2. A band-limited signal of finite energy, which has no frequency components higher 
than W Hertz, may be completely recovered from a knowledge of its samples taken. 
at the rate of 2W samples per second. 
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FIGURE 3.3 (a) Spectrum of a signal. (b) Spectrum of an undersampled version of the signal 
exhibiting the aliasing phenomenon. 


The sampling rate of 2W samples per second, for a signal bandwidth of W Hertz, is called 
the Nyquist rate; its reciprocal 1/2W (measured in seconds) is called the Nyquist interval. 

The derivation of the sampling theorem, as described herein, is based on the as- 
sumption that the signal g(ż) is strictly band limited. In practice, however, an information- 
bearing signal is not strictly band limited, with the result that some degree of undersam- 
pling is encountered. Consequently, some aliasing is produced by the sampling process. 
Aliasing refers to the phenomenon of a high-frequency component in the spectrum of the 
signal seemingly taking on the identity of a lower frequency in the spectrum of its sampled 
version, as illustrated in Figure 3.3. The aliased spectrum, shown by the solid curve in 
Figure 3.3b, pertains to an “undersampled” version of the message signal represented by 
the spectrum of Figure 3.32. 

To combat the effects of aliasing in practice, we may use two corrective measures, 
as described here: 


1. Prior, to sampling, a low-pass anti-aliasing filter is used to attenuate those high- 
frequency components of the signal that are not essential to the information being 
conveyed by the signal. 

2. The filtered signal is sampled at a rate slightly higher than the Nyquist rate. 


The use of a sampling rate higher than the Nyquist rate also has the beneficial effect of 
easing the design of the reconstruction filter used to recover the original signal from its 
sampled version. Consider the example of a message signal that has been anti-alias (low- 
pass) filtered, resulting in the spectrum shown in Figure 3.4a. The corresponding spectrum 
of the instantaneously sampled version of the signal is shown in Figure 3.40, assuming a 
sampling rate higher than the Nyquist rate. According to Figure 3.4b, we readily see that 
the design of the reconstruction filter may be specified as follows (see Figure 3.4c): 


> The reconstruction filter is low-pass with a passband extending from -W to W, 
which is itself determined by the anti-aliasing filter. 

> The filter has a transition band extending (for positive frequencies) from W to f, — W, 
where f, is the sampling rate. 
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FIGURE 3.4 (a) Anti-alias filtered spectrum of an information-bearing signal. (b) Spectrum of 
instantaneously sampled version of the signal, assuming the use of a sampling rate greater than 
the Nyquist rate. (c) Magnitude response of reconstruction filter. 


The fact that the reconstruction filter has a well-defined transition band means that it is 
physically realizable. 


i 3.3 Pulse-Amplitude Modulation 


Now that we understand the essence of the sampling process, we are ready to formally 
define pulse-amplitude modulation, which is the simplest and most basic form of analog 
pulse modulation. In pulse-amplitude. modulation (PAM), the amplitudes of regularly 
spaced pulses are varied in proportion to the corresponding sample values of a continuous 
message signal; the pulses can be of a rectangular form or some other appropriate shape. 
Pulse-amplitude modulation as defined here is somewhat similar to natural sampling, 
where the message signal is multiplied by a periodic train of rectangular pulses. However, 
in natural sampling the top of each modulated rectangular pulse varies with the messag? 
signal, whereas in PAM it is maintained flat; natural sampling is explored further in Prob- 
lem 3.2. 

The waveform of a PAM signal is illustrated in Figure 3.5. The dashed curve in this 
figure depicts the waveform of a message signal m(t), and the sequence of amplitude- 


3.3 Pulse-Amplitude Modulation 189 





FIGURE 3.5 Flat-top samples, representing an analog signal. 


modulated rectangular pulses shown as solid lines represents the corresponding PAM sig- 
nal s(t). There are two operations involved in the generation of the PAM signal: 


1. Instantaneous sampling of the message signal m(t) every T, seconds, where the sam- 
pling rate f, = 1/T, is chosen in accordance with the sampling theorem. 


2. Lengthening the duration of each sample so obtained to some constant value T. 


In digital circuit technology, these two operations are jointly referred to as “sample and 
hold.” One important reason for intentionally lengthening the duration of each sample is 
to avoid the use of an excessive channel bandwidth, since bandwidth is inversely propor- 
tional to pulse duration. However, care has to be exercised in how long we make the 
sample duration T, as the following analysis reveals. 

Let s(t) denote the sequence of flat-top pulses generated in the manner described in 
Figure 3.5. We may express the PAM signal as 


2 


st) = © m(nT,)b(t — nT,) (3.10) 


n= 


where T, is the sampling period and m(nT,) is the sample value of m(t) obtained at time 
t = nT,. The h(t) is a standard rectangular pulse of unit amplitude and duration T, defined 
as follows (see Figure 3.62): 


1, O<t<T 
ht)=4}3 t=0,t=T (3.11) 
0, otherwise 


By definition, the instantaneously sampled version of m(t) is given by 


© 


mt) = © m(nT,) (t— nT.) (3.12) 


where S(t — nT,) is a time-shifted delta function. Therefore, convolving m(t) with the 
pulse h(t), we get 


<0 


m,(t) *& h(t) = f m,rh(t — 7) dr 


n= 


= T Si m(nT,) (7 — nT,)h(t — 7) dt (3.13) 


It 


> m(nT) i 8(7 — nT,)h(t — 7) dr 


n= 
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Ficure 3.6 (a) Rectangular pulse h(t). (b) Spectrum H(f), made up of the magnitude |H(f), 
and phase arg[H(f)]. 


Using the sifting property of the delta function (see Appendix 2), we thus obtain 


æ 


m,(t) & b(t) = X m(nT,)b(t — nT) (3.14) 
From Equations (3.10) and (3.14) it follows that the PAM signal s(t) is mathematically 
equivalent to the convolution of 772,(t), the instantaneously sampled version of m(t), and 
the pulse (t), as shown by 


s(t) = m(t) * h(t) (3.15) 


Taking the Fourier transform of both sides of Equation (3.15) and recognizing that 
the convolution of two time functions is transformed into the multiplication of their re- 
spective Fourier transforms, we get 


S(f) = MAPHI) (3.16) 


where Sif) = F[s(t)], Maf) = Flm,(t)], and H(f) = F[b(t)|. Adapting Equation (3.2) to 
the problem at hand, we note that the Fourier transform M,(f) is related to the Fourier 
transform M(f) of the original message signal m(ż) as follows: 


MAP =f, D ME — kf) (3.17) 
where f, is the sampling rate. Therefore, substitution of Equation (3.17) into (3.16) yields 
Sif) =f 2 MY — RAHI) (3.18) 


Given a PAM signal s(t) whose Fourier transform S(f) is as defined in Equation 
(3.18), how do we recover the original message signal m(t)? As a first step in this reco 
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Figure 3.7 System for recovering message signal m(t) from PAM signal s(t), 


struction, we may pass s(t) through a low-pass filter whose frequency response is defined 
in Figure 3.4c; here it is assumed that the message is limited to bandwidth W and the 
sampling rate f, is larger than the Nyquist rate 2W. Then, from Equation (3.18) we find 
that the spectrum of the resulting filter output is equal to M(f)H(f). This output is equiv- 
alent to passing the original message signal m(t) through another low-pass filter of fre- 
quency response H(f). 5 

From Equation (3.11) we note that the Fourier transform of the rectangular pulse 
h(t) is given by 


H(f) = T sine(fT) exp(—jafT) (3.19) 


which is plotted in Figure 3.66. We see therefore that by using flat-top samples to generate 
a PAM signal, we have introduced amplitude distortion as well as a delay of T/2. This 
effect is rather similar to the variation in transmission with frequency that is caused by the 
finite size of the scanning aperture in television. Accordingly, the distortion caused by the 
use of pulse-amplitude modulation to transmit an analog information-bearing signal is 
referred to as the aperture effect. 

This distortion may be corrected by connecting an equalizer in cascade with the low- 
pass reconstruction filter, as shown in Figure 3.7. The equalizer has the effect of decreasing 
the in-band loss of the reconstruction filter as the frequency increases in such a manner as 
to compensate for the aperture effect. Ideally, the magnitude response of the equalizer is 
given by 


1 o1 2 _ mf 
|H(f)| Tsinc(fT) sin(rfT) 


The amount of equalization needed in practice is usually small. Indeed, for a duty cycle 
T/T, = 0.1, the amplitude distortion is less than 0.5 percent, in which case the need for 
equalization may be omitted altogether. 

The transmission of a PAM signal imposes rather stringent requirements on the mag- 
nitude and phase responses of the channel, because of the relatively short duration of the 
transmitted pulses. Furthermore, the noise performance of a PAM system can never be 
better than baseband-signal transmission. Accordingly, we find that for transmission over 
long distances, PAM would be used only as a means of message processing for time- 
division multiplexing, from which conversion to some other form of pulse modulation is 
subsequently made; time-division multiplexing is discussed in Section 3.9. | 


| 3.4 Other Forms of Pulse Modulation 


In a pulse modulation system we may use the increased bandwidth consumed by the pulses 
to improve the noise performance of the system. This can be achieved by representing the 
sample values of the message signals by some property of the pulse other than amplitude: 





(3.20) 


> Pulse-duration modulation (PDM), also referred to as pulse-width modulation, 
where samples of the message signal are used to vary the duration of the individual 
pulses in the carrier. 
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> Pulse-position modulation (PPM), where the position of a pulse relative to its up. 
modulated time of occurrence is varied in accordance with the message signal, 


These two other forms of pulse modulation are illustrated in Figure 3.8 for the case of g 
sinusoidal modulating wave. i B 

In PDM, long pulses expend considerable power while bearing no additional infor. 
mation. If this unused power is subtracted from PDM so that only time transitions are 
preserved, we obtain PPM. Accordingly, PPM is a more efficient form of pulse modulation 
than PDM. 

Since in a PPM system the transmitted information is contained in the relative po. 
sitions of the modulated pulses, the presence of additive noise affects the performance of 
such a system by falsifying the time at which the modulated pulses are judged to occur, 
Immunity to noise can be established by making the pulse build up so rapidly that the 
time interval during which noise can exert any perturbation is very short. Indeed, additive 
noise would have no effect on the pulse positions if the received pulses were perfectly 
rectangular, because the presence of noise introduces only vertical perturbations. However, 
the reception of perfectly rectangular pulses would require an infinite channel bandwidth, 
which is of course impractical. Thus with a finite channel bandwidth in practice, we find 
that the received pulses have a finite rise time, so the performance of the PPM receiver is 
affected by noise, which is to be expected. 


m(t) 
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Fıcure 3.8 Illustrating two different forms of pulse-time modulation for the case of a sinusoi- 
dal modulating wave. (a) Modulating wave. (b) Pulse carrier. (c) PDM wave. (d) PPM wave. 
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As in a CW modulation system, the noise performance of a PPM system may be 
described in terms of the output signal-to-noise ratio (SNR). Also, to find the noise im- 
provement produced by PPM over baseband transmission of a message signal, we may use 
the figure of merit defined as the output signal-to-noise ratio of the PPM system divided 
by the channel signal-to-noise ratio; see Section 2.10. Assuming that the average power of 
the channel noise is small compared to the peak pulse power, the figure of merit of the 
PPM system is proportional to the square of the transmission bandwidth By normalized 
with respect to the message bandwidth W. When, however, the input signal-to-noise ratio 
drops below a critical value, the system suffers a loss of the wanted message signal at the 
receiver output. That is, a PPM system suffers from a threshold effect of its own. 


3.5 Bandwidth—Noise Trade-Off 


In the context of noise performance, a PPM system is the optimum form of analog pulse 
modulation. The noise analysis of a PPM system reveals that pulse-position modulation 
(PPM) and frequency modulation (FM) systems exhibit a similar noise performance, as 
summarized here.' 


1. Both systems have a figure of merit proportional to the square of the transmission 
bandwidth normalized with respect to the message bandwidth. 


2. Both systems exhibit a threshold effect as the signal-to-noise ratio is reduced. 


The practical implication of point 1 is that, in terms of a trade-off of increased transmission 
bandwidth for improved noise performance, the best that we can do with continuous-wave 
(CW) modulation and analog pulse modulation systems is to follow a square law. A ques- 
tion that arises at this point in the discussion is: Can we produce a trade-off better than a 
square law? The answer is an emphatic yes, and digital pulse modulation is the way to do 
it. The use of such a method is a radical departure from CW modulation. 

Specifically, in a basic form of digital pulse modulation known as pulse-code mod- 
ulation (PCM),” a message signal is represented in discrete form in both time and ampli- 
tude. This form of signal representation permits the transmission of the message signal as 
a sequence of coded binary pulses. Given such a sequence, the effect of channel noise at 
the receiver output can be reduced to a negligible level simply by making the average power 
of the transmitted binary PCM wave large enough compared to the average power of the 
noise. 

Two fundamental processes are involved in the generation of a binary PCM wave: 
sampling and quantization. The sampling process takes care of the discrete-time represen- 
tation of the message signal; for its proper application, we have to follow the sampling 
theorem described in Section 3.2. The quantization process takes care of the discrete- 
amplitude representation of the message signal; quantization is a new process, the details 
of which are described in the next section. For now it suffices to say that the combined 
use of sampling and quantization permits the transmission of a message signal in coded 
form. This, in turn, makes it possible to realize an exponential law for the bandwidth- 
noise trade-off, which is also demonstrated in the next section. 


13.6 Quantization Process? 


A continuous signal, such as voice, has a continuous range of amplitudes and therefore its 
samples have a continuous amplitude range. In other words, within the finite amplitude 
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FIGURE 3.9 Description of a memoryless quantizer. 


range of the signal, we find an infinite number of amplitude levels. It is not necessary in 
fact to transmit the exact amplitudes of the samples. Any human sense (the ear or the eye), 
as ultimate receiver, can detect only finite intensity differences. This means that the original 
continuous signal may be approximated by a signal constructed of discrete amplitudes 
selected on.a minimum error basis from an available set. The existence of a finite number 
of discrete amplitude levels is a basic condition of pulse-code modulation. Clearly, if we 
assign the discrete amplitude levels with sufficiently close spacing, we may make the ap. 
proximated signal practically indistinguishable from the original continuous signal. 

Amplitude quantization is defined as the process of transforming the sample ampli- 
tude m(nT,) of a message signal m(t) at time t = nT, into a discrete amplitude v (nT,) taken 
from a finite set of possible amplitudes. We assume that the quantization process is 
memoryless and instantaneous, which means that the transformation at time ¢ = nT, is 
not affected by earlier or later samples of the message signal. This simple form of scalar 
quantization, though not optimum, is commonly used in practice. 

When dealing with a memoryless quantizer, we may simplify the notation by drop- 
ping the time index, We may thus use the symbol m in place of m(nT,), as indicated in the 
block diagram of a quantizer shown in Figure 3.94. Then, as shown in Figure. 3.90, the 
signal amplitude m is specified by the index k if it lies inside the partition cell 


fr: fng <m= My+1hs k = 1, 2, cb ee L (3.21) 


where L is the total number of amplitude levels used in the quantizer. The discrete ampli- 
tudes m, k = 1, 2,..., L, at the quantizer input are called decision levels or decision 
thresholds. At the quantizer output, the index k is transformed into an amplitude v, that 
represents all amplitudes of the cell $,; the discrete amplitudes v}, k = 1, 2,..., L, are 
called representation levels or reconstruction levels, and the spacing between two adjacent 
representation levels is called a quantum or step-size. Thus, the quantizer output v equals 
vp if the input signal sample m belongs to the interval $,. The mapping (see Figure 3,9a) 


v = g(m) (3.22) 


is the quantizer characteristic, which is a staircase function by definition. 

Quantizers can be of a uniform or nonuniform type. In a uniform quantizer, the 
representation levels are uniformly spaced; otherwise, the quantizer is nonuniform. In this 
section, we consider only uniform quantizers; nonuniform quantizers are considered in 
Section 3.7. The quantizer characteristic can also be of midtread or midrise type. Figure 
3.10a shows the input-output characteristic of a uniform quantizer of the midtread type, 
which is so called because the origin lies in the middle of a tread of the staircaselike graph. 
Figure 3.10b shows the corresponding input-output characteristic of a uniform quantizel 
of the midrise type, in which the origin lies in the middle of a rising part of the staircaselike 
graph. Note that both the midtread and midrise types of uniform quantizers illustrated in 
Figure 3.10 are symmetric about the origin. 
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FIGURE 3.10 Two types of quantization: (a) midtread and (b) midrise. 


E QUANTIZATION NOISE 


The use of quantization introduces an error defined as the difference between the input 
signal m and the output signal v. The error is called quantization noise. Figure 3.11 illus- 
trates a typical variation of the quantization noise as a function of time, assuming the use 
of a uniform quantizer of the midtread type. 

Let the quantizer input be the sample value of a zero-mean random variable M. 
(If the input has a nonzero mean, we can always remove it by subtracting the mean from 
the input and then adding it back after quantization.) A quantizer g(:) maps the input 
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FIGURE 3.11 Illustration of the quantization process. (Adapted from Bennett, 1948, with 
permission of AT&T.) 
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tandom variable M of continuous amplitude into a discrete random variable V; thei 
respective sample values 7m and v are related by Equation (3.22). Let the quantization erro, 
be denoted by the random variable Q of sample value q. We may thus write 


q=m-v (3.23) 
or, correspondingly, 

Q=M-V (3.24) 
With the input M having zero mean, and the quantizer assumed to be symmetric as in 
Figure 3.10, it follows that the quantizer output V and therefore the quantization error 
Q, will also have zero mean. Thus for a partial statistical characterization of the quantizer 
in terms of output signal-to-(quantization) noise ratio, we need only find the mean-square 
value of the quantization error Q. 

Consider then an input of continuous amplitude in the range (~mMmax max), 

Assuming a uniform quantizer of the midrise type illustrated in Figure 3.106, we find that 
the step-size of the quantizer is given by 


2M max 
A= (3.25) 
where L is the total number of representation levels. For a uniform quantizer, the quan- 
tization error Q will have its sample values bounded by —A/2 = q = A/2. If the step-size 
A is sufficiently small (i.e., the number of representation levels L is sufficiently large), it is 
reasonable to assume that the quantization error Q is a uniformly distributed random 
variable, and the interfering effect of the quantization noise on the quantizer input is similar 
to that of thermal noise. We may thus express the probability density function of the 
quantization error Q as follows: 
folg)= ya? 23172 (3.26) 
0, otherwise 


For this to be true, however, we must ensure that the incoming signal does not overload 
the quantizer. Then, with the mean of the quantization error being zero, its variance ob 
is the same as the mean-square value: 


oh = E{Q’] (3.27) 


A/2 
Ta ee T fola) 4q 


Substituting Equation (3.26) into (3.27), we get 


1 AZ 
D= | f dq 
cote (3.28) 


TR 


Typically, the L-ary number k, denoting the kth representation level of the quantizet, 
is transmitted to the receiver in binary form. Let R denote the number of bits per samp"? 
used in the construction of the binary code, We may then write 


L=28 (3.29) 
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or, equivalently, 
R = logs L (3.30) 


Hence, substituting Equation (3.29) into (3.25), we get the step size 


2 
A= =g (3.31) 

Thus the use of Equation (3.31) in (3.28) yields 
OB = Mma 7 (3.32) 


Let P denote the average power of the message signal m(t). We may then express the output 
signal-to-noise ratio of a uniform quantizer as 


P 
(SNR)o = = 


3P 
Ja 232R 
(=) 


Equation (3.33) shows that the output signal-to-noise ratio of the quantizer increases ex- 
ponentially with increasing number of bits per sample, R. Recognizing that an increase in 
R requires a proportionate increase in the channel (transmission) bandwidth Bzy, we thus 
see that the use of a binary code for the representation of a message signa! (as in pulse- 
code modulation) provides a more efficient method than either frequency modulation (FM) 
or pulse-position modulation (PPM) for the trade-off of increased channel bandwidth for 
improved noise performance. In making this statement, we presume that the FM and PPM 
systems are limited by receiver noise, whereas the binary-coded modulation system is lim- 
ited by quantization noise. We have more to say on the latter issue in Section 3.8. 


(3.33) 





> ExaMPLE 3.1 Sinusoidal Modulating Signal 


Consider the special case of a full-load sinusoidal modulating signal of amplitude A,,, which 
utilizes all the representation levels provided. The average signal power is (assuming a load 
of 1 ohm) 


The total range of the quantizer input is 2A,,, because the modulating signal swings between 
—A,, and Am. We may therefore set mazs = Am, in which case the use of Equation (3.32) 
yields the average power (variance) of the quantization noise as 


= 1429-2K 
OG = $472 
Thus the output signal-to-noise ratio of a uniform quantizer, for a full-load test tone, is 


A2/2 3 


(SNR)o = 2 2 (2?) (3.34) 


Expressing the signal-to-noise ratio in decibels, we get 


10 logio(SNR)o = 1.8 + 6R (3.35) 
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TABLE 3.1  Signal-to-(quantization) noise ratio 
for varying number of representation levels 
for sinusoidal modulation 





Number of 
Number of Representation Bits per Signal-to-Noise 
Levels, L Sample, R Ratio (dB) 
32 5 31.8 
64 6 37.8 
128 7 43.8 
256 8 49.8 





For various values of L and R, the corresponding values of signal-to-noise ratio are as given 
in Table 3.1. From Table 3.1 we can make a quick estimate of the number of bits per sample 
required for a desired output signal-to-noise ratio, assuming sinusoidal modulation. <4 


Thus far in this section we have focused on how to characterize memoryless scalar 
quantizers and assess their performance. In so doing, however, we avoided the optimum 
design of quantizers, that is, the issue of selecting the representation levels and partition 
cells so as to minimize the average quantization power for a prescribed number of repre- : 
sentation levels. Unfortunately, this optimization problem does not lend itself to a closed- ` 
form solution because of the highly nonlinear nature of the quantization process. Rather, _ 
we have effective algorithms for finding the optimum design in an iterative manner. A well- 
known algorithm that deserves to be mentioned in this context is the Lloyd-Max quantizer, ' 


which is discussed next. 


= CONDITIONS FOR OPTIMALITY OF SCALAR QUANTIZERS 


In designing a scalar quantizer the challenge is how to select the representation levels and 
surrounding partition cells so as to minimize the average quantization power for a fixed’ 
number of representation levels. 

To state the problem in mathematical terms, consider a message signal m(t) drawn 
from a stationary process M(t). Let -A =m =A denote the dynamic range of m(t), which 
is partitioned into a set of L cells, as depicted in Figure 3.12. The boundaries of the 
partition cells are defined by a set of real numbers 1, 12,..-, ML+1 that’ satisfy the 
following three conditions: 


m =-åÅ 
Msı = A 
m, S Mya fork = 1,2,...,L 


The &th partition cell is defined by 








Gut my < m £ myers for k = 1, 2,..., L (3.36) 
| l eee eal L J 

mç=-å m m3 m,-1 m, m4, =tA 
I 2A > 


FIGURE 3.12 Illustrating the partitioning of the dynamic range -A = m <A of a message 
signal m(t) into a set of L cells. 
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Let the representation levels (i.e., quantization values) be denoted by vg, k = 1,2,...,L 
Then, assuming that d(m, v,) denotes a.distortion measure for using v, to represent all 
those values of the input m that lie inside the partition cell $,, the goal is to find the two 
sets, {v,}-1 and {$,)}£-1, that minimize the average distortion 

L 


D=% Í dim, va)fulm) dm (3.37) 
k=l TME, 
where fm(m) is the probability density function of the random variable M with sample 
value m. 
A commonly used distortion measure is 


dim, v) = (m — vx) (3.38) 


in which case we speak of the mean-square distortion. In any event, the optimization 
problem stated herein is nonlinear, defying an explicit, closed-form solution. To get around 
this difficulty, we resort to an algorithmic approach for solving the problem in an iterative 
manner. 

Structurally speaking, the quantizer consists of two components with interrelated 
design parameters: 


> An encoder characterized by the set of partition cells {¥,}{-,; it is located in the 
transmitter. 

> A decoder characterized by the set of representation levels {v,}£_,; it is located in the 
receiver. 


Accordingly, we may identify two critically important conditions that provide the math- 
ematical basis for all algorithmic solutions to the optimum quantization problem. One 
condition assumes that we are given a decoder and the problem is to find the optimum 
encoder in the transmitter. The other condition assumes that we are given an encoder and 
the problem is to find the optimum decoder in the receiver. Henceforth, these two condi- 
tions are referred to as condition I and condition II, respectively. 


Condition I. Optimality of the Enceder for a Given Decoder 





The availability of a decoder means that we have a certain codebook in mind. Let the 
codebook be defined by 


(Usha (3.39) 


Given the codebook ©, the problem is to find the set of partition cells {,}£., that mini- 
mizes the average distortion D. That is, we wish to find the encoder defined by the non- 
linear mapping 


gim) =v, k=1,2,...,L (3.40) 
such that we have 
D= is dim, glm))fulm jamz > | [min d(m, v,)]fulmm) dm (3.41) 
MEF vpEg 


For the lower bound specified in Equation (3.41) to be attained, we require that the non- 
linear mapping of Equation (3.40) be satisfied only if the condition 


d(m, vy) = d(m, v) holds for all j # k (3.42) 
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The necessary condition described in Equation (3.42) for optimality of the encoder for a 
specified codebook is recognized as the nearest neighbor condition. In words, the nearest 
neighbor condition requires that the partition cell $, should embody all those values of 
the input 7 that are closer to vg than any other element of the codebook C. This optimality 
condition is indeed intuitively satisfying. 


Condition Il. Optimality of the Decoder for a Given Encoder 


Consider next the reverse situation to that described under condition I, which may be 
stated as follows: Optimize the codebook € = {v,}b-1 for the decoder, given that the set 
of partition cells {$ a\ke1 characterizing the encoder is fixed. The criterion for optimization 
is the average (mean-square) distortion: 


L 
D=% Í (m — v4) fulm) dm (3.43) 
ÉZ1 AMEI 

The probability density function fm(7) is clearly independent of the codebook 6. Hence, 
differentiating D with respect to the representation level v,, we readily obtain, 

áD Z 

w -2 5 [ (m — vg) fy(m) dm (3.44) 

Vk k=1 “MESIR 


Setting dD/dv, equal to zero and then solving for Yg, we obtain the optimum value 


J, mhaim) dm 
vron = ——— (3.45) 


es fulm) dm 


The denominator in Equation (3.45) is just the probability, px, that the random variable 
M with sample value m lies in the partition cell $, as shown by 


pr = Pim, < M = m, + 1) 


(3.46) 
= fl ful) dm 


Accordingly, we may interpret the optimality condition of Equation (3.45) as choosing 
the representation level vg to equal the conditional mean of the random variable M, given 
that M lies in the partition cell $4. We can thus formally state the condition for optimality 
of the decoder for a given encoder as follows: 


Vk opt = E[M|m, < M = menl (3.47) 


where E is the expectation operator. Equation (3.47) is also intuitively satisfying. 

Note that the nearest neighbor condition (condition I) for optimality of the encoder 
for a given decoder was proved for a generic average distortion. However, the conditional 
mean requirement (condition II) for optimality of the decoder for a given encoder was 
proved for the special case of a mean-square distortion. In any event, these two conditions 
are necessary for optimality of a scalar quantizer. Basically, the algorithm for designing 
the quantizer consists of alternately optimizing the encoder in accordance with condition 
I, then optimizing the decoder in accordance with condition II, and continuing in this 
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manner until the average distortion D reaches a minimum. An optimum quantizer designed 
in this manner is called a Lloyd-Max quantizer.* 


l 3.7 Pulse-Code Modulation 


With the sampling and quantization processes at our disposal, we are now ready to de- 
scribe pulse-code modulation, which, as mentioned previously, is the most basic form of 
digital pulse modulation. In pulse-code modulation (PCM), a message signal is represented 
by a sequence of coded pulses, which is accomplished by representing the signal in discrete 
form in both time and amplitude. The basic operations performed in the transmitter of a 
. PCM system are sampling, quantizing, and encoding, as shown in Figure 3.134; the low- 
pass filter prior to sampling is included to prevent aliasing of the message signal. The 
quantizing and encoding operations are usually performed in the same circuit, which is 
called an analog-to-digital converter. The basic operations in the receiver are regeneration 
of impaired signals, decoding, and reconstruction of the train of quantized samples, as 
shown in Figure 3.13c. Regeneration also occurs at intermediate points along the trans- 
mission path as necessary, as indicated in Figure 3.13b. When time-division multiplexing 
is used, it becomes necessary to synchronize the receiver to the transmitter for the overall 
system to operate satisfactorily, as discussed in Section 3.9. In what follows, we describe 
the various operations that constitute a basic PCM system. 


u SAMPLING 


The incoming message signal is sampled with a train of narrow rectangular pulses so as 
to closely approximate the instantaneous sampling process. To ensure perfect reconstruc- 
tion of the message signal at the receiver, the sampling rate must be greater than twice the 
highest frequency component W of the message signal in accordance with the sampling 
theorem. In practice, a low-pass anti-aliasing filter is used at the front end of the sampler 
to exclude frequencies greater than W before sampling. Thus the application of sampling 
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FIGURE 3.13 The basic elements of a PCM system. 
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permits the reduction of the continuously varying message signal (of some finite duration) 
to a limited number of discrete values per second. 


QUANTIZATION 


The sampled version of the message signal is then quantized, thereby providing a new 
representation of the signal that is discrete in both time and amplitude. The quantization 
process may follow a uniform law as described in Section 3.6. In telephonic communica- 
tion, however, it is preferable to use a variable separation between the representation levels, 
For example, the range of voltages covered by voice signals, from the peaks of loud talk 
to the weak passages of weak talk, is on the order of 1000 to 1. By using a nonuniform 
quantizer with the feature that the step-size increases as the separation from the origin of 
the input-output amplitude characteristic is increased, the large end steps of the quantizer 
can take care of possible excursions of the voice signal into the large amplitude ranges 
that occur relatively infrequently. In other words, the weak passages, which need more 
protection, are favored at the expense of the loud passages. In this way, a nearly uniform 
percentage precision is achieved throughout the greater part of the amplitude range of the 
input signal, with the result that fewer steps are needed than would be the case if a uniform 
quantizer were used. 

The use of a nonuniform quantizer is equivalent to passing the baseband signal 
through a compressor and then applying the compressed signal to a uniform quantizer. A 
particular form of compression law that is used in practice is the so-called p-law,* which 
is defined by . 


_ log + w|m|) 
|v] = 
log(1 + y) 
where m and v are the normalized input and output voltages, and yu is a positive constant. 


In Figure 3.14, we have plotted the y-law for three different values of p. The case of 
uniform quantization corresponds to u = 0. For a given value of u, the reciprocal slope 


(3.48). 
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Ficure 3.14 Compression laws. (a) p-law. (b) A-law. 
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of the compression curve, which defines the quantum steps, is given by the derivative of 
|| with respect to |v|; that is, 
d|m| _ log(1 +4) 
d\v| 
We see therefore that the y-law is neither strictly linear not strictly logarithmic, but it is 
approximately linear at low input levels corresponding to u |m| << 1, and approximately 
logarithmic at high input levels corresponding to | m| >> 1. 
Another compression law that is used in practice is the so-called A-law defined by 
A|m| 
1 + log A’ 
1 + log(A|m]|) 1 
1+logA > A 


(1 + ||) (3.49) 


1 
<= eee 
0<|m|< 


|v| = (3.50) 





which is plotted in Figure 3.146 for varying A. The case of uniform quantization corre- 
sponds to A = 1. The reciprocal slope of this second compression curve is given by 
the derivative of |m] with respect to |v|, as shown by (depending on the value assigned 
to the normalized input |7|) 





d|m| A A 
div] = 1 (3.51) 
(1+ A)lm|, 2 <|m| <1 


To restore the signal samples to their correct relative level, we must, of course, use 
a device in the receiver with a characteristic complementary to the compressor. Such a 
device is called an expander. Ideally, the compression and expansion laws are exactly 
inverse so that, except for the effect of quantization, the expander output is equal to the 
compressor input. The combination of a compressor and an expander is called a 
compander. 

For both the u-law and A-law, the dynamic range capability of the compander im- 
proves with increasing u and A, respectively. The SNR for low-level signals increases at 
the expense of the SNR for high-level signals. To accommodate these two conflicting 
requirements (i.e., a reasonable SNR for both low- and high-level signals), a compromise 
is usually made in choosing the value of paramenter p for the -law and parameter A for 
the A-law. The typical values used in practice are: 4 = 255 and A = 87.6. 

It is also of interest to note that in actual PCM systems, the companding circuitry 
does not produce an exact replica of the nonlinear compression curves shown in Figure 
3.14. Rather, it provides a piecewise linear approximation to the desired curve. By using 
a large enough number of linear segments, the approximation can approach the true com- 
pression curve very closely. This form of approximation is illustrated in Example 3.2. 


m ENCODING 


In combining the processes of sampling and quantization, the specification of a continuous 
message (baseband) signal becomes limited to a discrete set of values, but not in the form 
best suited to transmission over a telephone line or radio path. To exploit the advantages 
of sampling and quantizing for the purpose of making the transmitted signal more robust 
to noise, interference and other channel impairments, we require the use of an encoding 
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TABLE 3.2 Binary number system 
for R = 4 bits/sample 


Ordinal Number of Level Number Expressed as Binary 





Representation Level Sum of Powers of 2 Number 
0 0000 
1 2° 0001 
2 21 0010 
3 2! + 2° 0011 
4 22 0100 
5 2? +2° 0101 
6 2? +21 0110 
7 274+ 214+ 2° 0111 
8 23 1000 
9 23 +2° 1001 

10 2 +2! 1010 
11 2 +2+4+ 2° 1011 
12 23 +2? 1100 
13 2+2 +2° 1101 
14 23 + 2% + 21 1110 
15 23+ 2?4+ 214+ 2° 1111 





process to translate the discrete set of sample values to a more appropriate form of signal, 
Any plan for representing each of this discrete set of values as a particular arrangement of 
discrete events is called a code. One of the discrete events in a code is called a code element 
or symbol. For example, the presence or absence of a pulse is a symbol. A particular 
arrangement of symbols used in a code to represent a single value of the discrete set is 
called a code word or character. 

In a binary code, each symbol may be either of two distinct values or kinds, such as 
the presence or absence of a pulse. The two symbols of a binary code are customarily 
denoted as 0 and 1. In a ternary code, each symbol may be one of three distinct values or 
kinds, and so on for other codes. However, the maximum advantage over the effects of 
noise in a transmission medium is obtained by using a binary code, because a binary 
symbol withstands a relatively high level of noise and is easy to regenerate. Suppose that, 
in a binary code, each code word consists of R bits: bit is an acronym for binary digit; 
thus R denotes the number of bits per sample. Then, using such a code, we may represent 
a total of 2? distinct numbers. For example, a sample quantized into one of 256 levels 
may be represented by an 8-bit code word. 

There are several ways of establishing a one-to-one correspondence between repre- 
sentation levels and code words. A convenient method is to express the ordinal number 
of the representation level as a binary number. In the binary number system, each digit 
has a place-value that is a power of 2, as illustrated in Table 3.2 for the case of four bits 
per sample (i.e., R = 4). 


Line Codes 


Any of several line codes can be used for the electrical representation of a binary 
data stream. Figure 3.15 displays the waveforms of five important line codes for the ex 
ample data stteam 01101001. Figure 3.16 displays their individual power spectra (tor 
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Binary data 0 1 1 ie} 1 0 0 $ 
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le) 
FIGURE 3.15 Line codes for the electrical representations of binary data. (a) Unipolar NRZ 
signaling. (b) Polar NRZ signaling. (c) Unipolar RZ signaling. (d) Bipolar RZ signaling. 
(e) Split-phase or Manchester code. 


positive frequencies) for randomly generated binary data, assuming that (1) symbols 0 and 
1 are equiprobable, (2) the average power is normalized to unity, and (3) the frequency f 
is normalized with respect to the bit rate 1/T,, (For the formulas used to plot the power 
spectra of Figure 3.16, the reader is referred to Problem 3.11.) The five line codes illustrated 
in Figure 3.15 are described here: 


1. Unipolar nonreturn-to-zero (NRZ) signaling 

In this line code, symbol 1 is represented by transmitting a pulse of amplitude A for the 
duration of the symbol, and symbol 0 is represented by switching off the pulse, as in Figure 
3.15a. This line code is also referred to as on-off signaling. Disadvantages of on-off sig- 
naling are the waste of power due to the transmitted DC level and the fact that the power 
spectrum of the transmitted signal does not approach zero at zero frequency. 


2. Polar nonreturn-to-zero (NRZ) signaling 

In this second line code, symbols 1 and 0 are represented by transmitting pulses of ampli- 
tudes +A and —A, respectively, as illustrated in Figure 3.155. This line code is relatively 
easy to generate but its disadvantage is that the power spectrum of the signal is large near 
zero frequency. 
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FIGURE 3.16 Power spectra of line codes: (a) Unipolar NRZ signal. (b) Polar NRZ signal. 
(c) Unipolar RZ signal. (d) Bipolar RZ signal. (e) Manchester-encoded signal. The frequency is 
normalized with respect to the bit rate 1/T,, and the average power is normalized to unity. 
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3. Unipolar return-to-zero (RZ) signaling 

In this other line code, symbol 1 is represented by a rectangular pulse of amplitude A and 
half-symbol width, and symbol 0 is represented by transmitting no pulse, as illustrated in 
Figure 3.15c. An attractive feature of this line code is the presence of delta functions at 
f = 0, +1/T, in the power spectrum of the transmitted signal, which can be used for bit- 
timing recovery at the receiver. However, its disadvantage is that it requires 3 dB more 
power than polar return-to-zero signaling for the same probability of symbol error; this 
issue is addressed in Chapter 4 under Problem 4.10. 


4. Bipolar return-to-zero (BRZ) signaling 

This line code uses three amplitude levels as indicated in Figure 3.15d. Specifically, positive 
and negative pulses of equal amplitude (i.e., +A and —A) are used alternately for symbol 
1, with each pulse having a half-symbol width; no pulse is always used for symbol 0. A 
useful property of the BRZ signaling is that the power spectrum of the transmitted signal 
has no DC component and relatively insignificant low-frequency components for the case 
when symbols 1 and 0 occur with equal probability. This line code is also called alternate 
mark inversion (AMI) signaling. 


5. Split-phase (Manchester code) 

In this method of signaling, illustrated in Figure 3.15e, symbol 1 is represented by a positive 
pulse of amplitude A followed by a negative pulse of amplitude —A, with both pulses being 
half-symbol wide. For symbol 0, the polarities of these two pulses are reversed. The Man- 
chester code suppresses the DC component and has relatively insignificant low-frequency 
components, regardless of the signal statistics. This property is essential in some 
applications. 


Differential Encoding 


This method is used to encode information in terms of signal transitions. In partic- 
ular, a transition is used to designate symbol 0 in the incoming binary data stream, while 
No transition is used to designate symbol 1, as illustrated in Figure 3.17. In Figure 3.175 
we show the differentially encoded data stream for the example data specified in Figure 
3.174. The original binary data stream used here is the same as that used in Figure 3.15. 
The waveform of the differentially encoded data is shown in Figure 3.17c, assuming the 
use of unipolar nonreturn-to-zero signaling. From Figure 3.17 it is apparent that a differ- 
entially encoded signal may be inverted without affecting its interpretation. The original 
binary information is recovered simply by comparing the polarity of adjacent binary sym- 
bols to establish whether or not a transition has occurred. Note that differential encoding 
requires the use of a reference bit before initiating the encoding process. In Figure 3.17, 
symbol 1 is used as the reference bit. 


(a) Original binary data 0 1 1 0 1 0 60 1 


{b} Differentially encoded data 1 ie) 0 0 1 1 0 1 1 


(c) Waveform 
Reference bit 9 


Time —> 


FIGURE 3.17 (a) Original binary data. (b) Differentially encoded data, assuming reference bit 1. 
(c) Waveform of differentially encoded data using unipolar NRZ signaling. 
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FIGURE 3.18 Block diagram of regenerative repeater. 


REGENERATION 


The most important feature of PCM systems lies in the ability to control the effects of 
distortion and noise produced by transmitting a PCM signal through a channel. This 
capability is accomplished by reconstructing the PCM signal by means of a chain of re- 
generative repeaters located at sufficiently close spacing along the transmission route. As 
illustrated in Figure 3.18, three basic functions are performed by a regenerative repeater; 
equalization, timing, and decision making. The equalizer shapes the received pulses so as 
to compensate for the effects of amplitude and phase distortions produced by the nonideal 
transmission characteristics of the channel. The timing circuitry provides a periodic pulse 
train, derived from the received pulses, for sampling the equalized pulses at the instants of 
time where the signal-to-noise ratio is a maximum. Each sample so extracted is compared 
to a predetermined threshold in the decision-making device. In each bit interval, a decision 
is then made whether the received symbol is a 1 or a 0 on the basis of whether the threshold 
is exceeded or not. If the threshold is exceeded, a clean new pulse representing symbol 
1 is transmitted to the next repeater. Otherwise, another clean new pulse representing 
symbol 0 is transmitted. In this way, the accumulation of distortion and noise in a repeater 
span is completely removed, provided that the disturbance is not too large to cause an 
error in the decision-making process. Ideally, except for delay, the regenerated signal is 
exactly the same as the signal originally transmitted, In practice, however, the regenerated 
signal departs from the original signal for two main reasons: 


1, The unavoidable presence of channel noise and interference causes the repeater to 
make wrong decisions occasionally, thereby introducing bit errors into the regener- 
ated signal. 

2. If the spacing between received pulses deviates from its assigned value, a jitter is 
introduced into the regenerated pulse position, thereby causing distortion. 


a DECODING 


The first operation in the receiver is to regenerate (i.e, reshape and clean up) the received 
pulses one last time. These clean pulses are then regrouped into code words and decoded 
(i.e., mapped back) into a quantized PAM signal. The decoding process involves generating 
a pulse the amplitude of which is the linear sum of all the pulses in the code word, with 
each pulse being weighted by its place value (22,21, 2?,..., 2®-1) in the code, where Ris 
the number of bits per sample. 


= FILTERING 


The final operation in the receiver is to recover the message signal by passing the decoder 
ourput through a low-pass reconstruction filter whose cutoff frequency is equal to th? 
message bandwidth W. Assuming that the transmission path is error free, the recovered 


3.8 Noise Considerations in PCM Systems 209 


signal includes no noise with the exception of the initial distortion introduced by the 
quantization process. 





| 3-8 Noise Considerations in PCM Systems 


The performance of a PCM system is influenced by two major sources of noise: 


1. Channel noise, which is introduced anywhere between the transmitter output and 
the receiver input. Channel noise is always present, once the equipment is switched 
on. i 

2. Quantization noise, which is introduced in the transmitter and is carried all the way 
along to the receiver output. Unlike channel noise, quantization noise is signal- 
dependent in the sense that it disappears when the message signal is switched off. 


Naturally, these two sources of noise appear simultaneously once the PCM system is in 
operation. However, the traditional practice is to consider them separately, so that we may 
develop insight into their individual effects on the system performance. 

The main effect of channel noise is to introduce bit errors into the received signal. 
In the case of a binary PCM system, the presence of a bit error causes symbol 1 to be 
mistaken for symbol 0, or vice versa. Clearly, the more frequently bit errors occur, the 
more dissimilar the receiver output becomes compared to the original message signal. The 
fidelity of information transmission by PCM in the presence of channel noise may be 
measured in terms of the average probability of symbol error, which is defined as the 
probability that the reconstructed symbol at the receiver output differs from the transmit- 
ted binary symbol, on the average. The average probability of symbol error, also referred 
to as the bit error rate (BER), assumes that all the bits in the original binary wave are of 
equal importance. When, however, there is more interest in reconstructuring the analog 
waveform of the original message signal, different symbol errors may need to be weighted 
differently; for example, an error in the most significant bit in a code word (representing 
a quantized sample of the message signal) is more harmful than an error in the least 
significant bit. 

To optimize system performance in the presence of channel noise, we need to mini- 
mize the average probability of symbol error. For this evaluation, it is customary to model 
the channel noise as additive, white, and Gaussian. The effect of channel noise can be 
made practically negligible by ensuring the use of an adequate signal energy-to-noise den- 
sity ratio through the provision of short-enough spacing between the regenerative repeaters 
in the PCM system. In such a situation, the performance of the PCM system is essentially 
limited by quantization noise acting alone. 

From the discussion of quantization noise presented in Section 3.6, we recognize that 
quantization noise is essentially under the designer’s control. It can be made negligibly 
small through the use of an adequate number of representation levels in the quantizer and 
the selection of a companding strategy matched to the characteristics of the type of message 
signal being transmitted. We thus find that the use of PCM offers the possibility of building 
a communication system that is rugged with respect to channel noise on a scale that is 
beyond the capability of any CW modulation or analog pulse modulation system. 


= ERROR THRESHOLD 


The underlying theory of bit error rate calculation in a PCM system is deferred until 
Chapter 4. For the present, it suffices to say that the average probability of symbol error 
in a binary encoded PCM receiver due to additive white Gaussian noise depends solely on 
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E,/No, which is defined as the ratio of the transmitted signal energy per bit, Ep, to the 
noise spectral density, No. Note that the ratio E,/No is dimensionless even though the 
quantities E, and No have different physical meaning. In Table 3.3 we present a summary 
of this dependence for the case of a binary PCM system using polar nonreturn-to-zero 
signaling. The results presented in the last column of the table assume a bit rate of 10° bis, 

From Table 3.3 it is clear that there is an error threshold (at about 11 dB). For 
E,/No below the error threshold the receiver performance involves significant numbers of 
errors, and above it the effect of channel noise is practically negligible. In other words, 
provided that the ratio E,/No exceeds the error threshold, channel noise has virtually no 
effect on the receiver performance, which is precisely the goal of PCM. When, however, 
E,/No drops below the error threshold, there is a sharp increase in the rate at which errors 
occur in the receiver. Because decision errors result in the construction of incorrect code 
words, we find that when the errors are frequent, the reconstructed message at the receiver 
output bears little resemblance to the original message. 

Comparing the figure of 11 dB for the error threshold in a PCM system using polar 
NRZ signaling with the 60-70 dB required for high-quality transmission of speech using 
amplitude modulation, we see that PCM requires much less power, even though the av- 
erage noise power in the PCM system is increased by the R-fold increase in bandwidth, 
where R is the number of bits in a code word (i.e., bits per sample). 

In most transmission systems, the effects of noise and distortion from the individual 
links accumulate. For a given quality of overall transmission, the longer the physical sep- 
aration between the transmitter and the receiver, the more severe are the requirements on 
each link in the system. In a PCM system, however, because the signal can be regenerated 
as often as necessary, the effects of amplitude, phase, and nonlinear distortions in one link 
(if not too severe) have practically no effect on the regenerated input signal to the next 
link. We have also seen that the effect of channel noise can be made practically negligible 
by using a ratio E,/No above threshold. For all practical purposes, then, the transmission 
requirements for a PCM link are almost independent of the physical length of the com- 
munication channel. 

Another important characteristic of a PCM system is its ruggedness to interference, 
caused by stray impulses or cross-talk. The combined presence of channel noise and in- 
terference causes the error threshold necessary for satisfactory operation of the PCM sys- 
tem to increase. If an adequate margin over the error threshold is provided in the first 
place, however, the system can withstand the presence of relatively large amounts of in- 
terference. In other words, a PCM system is robust to channel noise and interference. 


TABLE 3,3 Influence of E,/No on the 








probability of error 
For a Bit Rate of 10° b/s, 
Probability of This Is About One 

E,/No Error P. Error Every 

4.3 dB 10°? 1073 second 

8.4 1074 107! second 
10.6 107° 10 seconds 
12.0 107° 20 minutes 
13.0 107° 1 day 


14.0 10-? 3 months 





3.9 Time-Division Multiplexing 211l 


l 3.9 Time-Division Multiplexing 


The sampling theorem provides the basis for transmitting the information contained in a 
band-limited message signal m(t) as a sequence of samples of m(t) taken uniformly at a 
rate that is usually slightly higher than the Nyquist rate. An important feature of the 
sampling process is a conservation of time. That is, the transmission of the message samples 
engages the communication channel for only a fraction of the sampling interval on a 
periodic basis, and in this way some of the time interval between adjacent samples is cleared 
for use by other independent message sources on a time-shared basis. We thereby obtain 
a time-division multiplex (TDM) system, which enables the joint utilization of a common 
communication channel by a plurality of independent message sources without mutual 
interference among them. 

The concept of TDM is illustrated by the block diagram shown in Figure 3.19. Each 
input message signal is first restricted in bandwidth by a low-pass anti-aliasing filter to 
remove the frequencies that are nonessential to an adequate signal representation. The 
low-pass filter outputs are then applied to a commutator, which is usually implemented 
using electronic switching circuitry. The function of the commutator is twofold: (1) to take 
a narrow sample of each of the N input messages at a rate f, that is slightly higher than 
2W, where W is the cutoff frequency of the anti-aliasing filter, and (2) to sequentially 
interleave these N samples inside the sampling interval T,. Indeed, this latter function is 
the essence of the time-division multiplexing operation. Following the commutation pro- 
cess, the multiplexed signal is applied to a pulse modulator, the purpose of which is to 
transform the multiplexed signal into a form suitable for transmission over the common 
channel. It is clear that the use of time-division multiplexing introduces a bandwidth ex- 
pansion factor N, because the scheme must squeeze N samples derived from N independent 
message sources into a time slot equal to one sampling interval. At the receiving end of 
the system, the received signal is applied to a pulse demodulator, which performs the 
reverse operation of the pulse modulator. The narrow samples produced at the pulse de- 
modulator output are distributed to the appropriate low-pass reconstruction filters by 
means of a decommutator, which operates in synchronism with the commutator in the 
transmitter. This synchronization is essential for a satisfactory operation of the system. 
The way this synchronization is implemented depends naturally on the method of pulse 
modulation used to transmit the multiplexed sequence of samples. 

The TDM system is highly sensitive to dispersion in the common channel, that is, to 
variations of amplitude with frequency or lack of proportionality of phase with frequency. 
Accordingly, accurate equalization of both magnitude and phase responses of the channel 
is necessary to ensure a satisfactory operation of the system; this issue is discussed in 
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FIGURE 3.19 Block diagram of TDM system. 
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Chapter 4. However, unlike FDM, to a first-order approximation TDM is immune to 
nonlinearities in the channel as a source of cross-talk. The reason for this behavior is that 
different message signals are not simultaneously applied to the channel. 


m SYNCHRONIZATION 


In applications using PCM, it is natural to multiplex different messages sources by time 
division, whereby each source keeps its individuality throughout the journey from the 
transmitter to the receiver. This individuality accounts for the comparative ease with which 
message sources may be dropped or reinserted in a time-division multiplex system. As the 
number of independent message sources is increased, the time interval that may be allotted 
to each source has to be reduced, since all of them must be accommodated into a time 
interval equal to the reciprocal of the sampling rate. This, in turn, means that the allowable 
duration of a code word representing a single sample is reduced. However, pulses tend to 
become more difficult to generate and to transmit as their duration is reduced. Further- 
more, if the pulses become too short, impairments in the transmission medium begin to 
interfere with the proper operation of the system. Accordingly, in practice, it is necessary 
to restrict the number of independent message sources that can be included within a time- 
division group. 

In any event, for a PCM system with time-division multiplexing to operate satisfac- 
torily, it is necessary that the timing operations at the receiver, except for the time lost in 
transmission and regenerative repeating, follow closely the corresponding operations at 
the transmitter. In a general way, this amounts to requiring a local clock at the receiver 
to keep the same time as a distant standard clock at the transmitter, except that the local 
clock is somewhat slower by an amount corresponding to the time required to transport 
the message signals from the transmitter to the receiver. One possible procedure to syn- 
chronize the transmitter and receiver clocks is to set aside a code element or pulse at the 
end of a frame (consisting of a code word derived from each of the independent message 
sources in succession) and to transmit this pulse every other frame only. In such a case, 
the receiver includes a circuit that would search for the pattern of 1s and 0s alternating at 
half the frame rate, and thereby establish synchronization between the transmitter and 
receiver. 

When the transmission path is interrupted, it is highly unlikely that transmitter and 
receiver clocks will continue to indicate the same time for long. Accordingly, in carrying 
out a synchronization process, we must set up an orderly procedure for detecting the 
synchronizing pulse. The procedure consists of observing the code elements one by one 
until the synchronizing pulse is detected. That is, after observing a particular code element 
long enough to establish the absence of the synchronizing pulse, the receiver clock is set 
back by one code element and the next code element is observed. This searching process 
is repeated until the synchronizing pulse is detected. Clearly, the time required for syn 
chronization depends on the epoch at which proper transmission is re-established. 


> EXAMPLE 3.2 The Tl System 


In this example, we describe the important characteristics of a PCM system known as the T! 
system,® which carries 24 voice channels over separate pairs of wires with regenerative rè 
peaters spaced at approximately 2-km intervals. The T1 carrier system is basic to the North 
American Digital Switching Hierarchy described in Section 3.10. , 
A voice signal (male or female) is essentially limited to a band from 300 to 3100 Ha 
that frequencies outside this band do not contribute much to articulation efficiency. Indeed, 
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telephone circuits that respond to this range of frequencies give quite satisfactory service. 
Accordingly, it is customary to pass the voice signal through a low-pass filter with a cutoff 
frequency of about 3.1 kHz prior to sampling. Hence, with W = 3.1 kHz, the nominal value 
of the Nyquist rate is 6.2 kHz. The filtered voice signal is usually sampled at a slightly higher 
rate, namely, 8 kHz, which is the standard sampling rate in telephone systems. 

For companding, the T1 system uses a piecewise-linear characteristic (consisting of 
15 linear segments) to approximate the logarithmic p-law of Equation (3.48) with the constant 
H = 255. This approximation is constructed in such a way that the segment end points lie on 
the compression curve computed from Equation (3.48), and their projections onto the vertical 
axis are spaced uniformly. Table 3.4 gives the projections of the segment end points onto the 
horizontal axis and the step-sizes of the individual segments. The table is normalized to 8159, 
so that all values are represented as integer numbers. Segment 0 of the approximation is a 
colinear segment, passing through the origin; it contains a total of 30 uniform decision levels, 
Linear segments 1a, 24,..., 7a lie above the horizontal axis, whereas linear segments 1b, 
2b,..., 7b lie below the horizontal axis; each of these 14 segments contains 16 uniform 
decision levels. For colinear segment 0. the decision levels at the quantizer input are +1, 
+3,..., £31, and the corresponding representation levels at the quantizer output are 0, 
+1,..., +15, For linear segments 14 and 1b, the decision levels at the quantizer input are 
+31, +35,..., +95, and the corresponding representation levels at the quantizer output are 
+16, +17,..., +31, and so on for the other linear segments. 

There are a total of 31 + (14 x 16) = 255 representation levels associated with the 
15-segment companding characteristic described above. To accommodate this number of tep- 
resentation levels, each of the 24 voice channels uses a binary code with an 8-bit word. The 
first bit indicates whether the input voice sample is positive or negative; this bit is a 1 if positive 
and a 0 if negative. The next three bits of the code word identify the particular segment inside 
which the amplitude of the input voice sample lies, and the last four bits identify the actual 
representation level inside that segment. 

With a sampling rate of 8 kHz, each frame of the multiplexed signal occupies a period 
of 125 ys. In particular, it consists of twenty-four 8-bit words, plus a single bit that is added 
at the end of the frame for the purpose of synchronization. Hence, each frame consists of a 
total of (24 X 8) + 1 = 193 bits, Correspondingly, the duration of each bit equals 0.647 ps, 
and the resulting transmission rate is 1.544 megabits per second (Mb/s). 

In addition to the voice signal, a telephone system must also Pass special supervisory 
signals to the far end. This signaling information is needed to transmit dial pulses, as well as 


i TABLE 3.4 The 15-segment companding characteristic (p = 255) 
Projections of Segment End Points 





























Linear Segment Number Step-Size onto the Horizontal Axis 

0 2 +31 
1a, 1b 4 +95 
2a, 2b 8 +223 
3a, 3b 16 +479 
4a, 4b 32 +991 
5a, 5b 64 +2015 
6a, 6b 128 +4063 





7a, 7b 256 +8159 
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telephone off-hook/on-hook signals. In the T1 system, this requirement is accomplished ag 
follows. Every sixth frame, the least significant (that is, the eighth) bit of each voice channel 
is deleted and a signaling bit is inserted in its place, thereby yielding an average 72-bit operation 
for each voice input. The sequence of signaling bits is thus transmitted at a rate equal to 
sampling rate of 8 kHz divided by six, that is, 1.333 kb/s. This signaling rate applies to each 
of the 24 input channels. 4 


I 3.10 Digital Multiplexers 


In Section 3.9 we introduced the idea of time-division multiplexing whereby a group of 
analog signals (e.g., voice signals) are sampled sequentially in time at a common sampling 
rate and then multiplexed for transmission over a common line. In this section we consider 
the multiplexing of digital signals at different bit rates. This enables us to combine several 
digital signals, such as computer outputs, digitized voice signals, digitized facsimile, 
and television signals, into a single data stream (at a considerably higher bit rate than 
any of the inputs). Figure 3.20 shows a conceptual diagram of the digital multiplexing. 
demultiplexing operation. 

The multiplexing of digital signals is accomplished by using a bit-by-bit interleaving 
procedure with a selector switch that sequentially takes a bit from each incoming line and 
then applies it to the high-speed common line. At the receiving end of the system the output 
of this common line is separated out into its low-speed individual components and then 
delivered to their respective destinations. 

Digital multiplexers are categorized into two major groups. One group of multiplex- 
ers is used to take relatively low bit-rate data streams originating from digital computers 
and multiplex them for TDM transmission over the public switched telephone net- 
work, The implementation of this first group of multiplexers requires the use of modems 
(modulators-demodulators), which are discussed in Chapter 6. 

The second group of digital multiplexers forms part of the data transmission service 
provided by telecommunication carriers such as AT&T. In particular, these multiplexers 
constitute a digital hierarchy that time-division multiplexes low-rate bit streams into much 
higher-rate bit streams. The details of the bit rates that are accommodated in the hierarchy 
vary from one country to another. However, a worldwide feature of the hierarchy is that 
it starts at 64 kb/s, which corresponds to the standard PCM representation of a voice 
signal. An incoming bit stream at this rate, irrespective of its origin, is called a digital signal 
zero (DSO). In the United States, Canada, and Japan’ the hierarchy follows the North 
American digital TDM hierarchy as described here: 


> The first-level hierarchy combines twenty-four DSO bit streams to obtain a digital 
signal one (DS1) at 1.544 Mb/s, which is carried on the T1 system described in 
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Ficure 3.20 Conceptual diagram of multiplexing-demultiplexing. 
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Example 3.2. These bit streams are called the primary rate in the digital hierarchy, 
because it is the lowest bit rate that exists outside a digital switch. The digital switch 
is a device consisting of memory and logic, the function of which is merely the switch- 
ing of digital signals, hence the name. 

> The second-level multiplexer combines four DS1 bit streams to obtain a digital signal 
two (DS2) at 6.312 Mb/s. 

> The third-level multiplexer combines seven DS2 bit streams to obtain a digital signal 
three (DS3) at 44.736 Mb/s. 7 

> The fourth-level multiplexer combines six DS3 bit streams to obtain a digital signal 
four (DS4) at 274.176 Mb/s. 

> The fifth-level multiplexer, the final one in the hierarchy, combines two DS4 bit 
streams to obtain a digital signal five (DSS) at 560.160 Mb/s. 


Note that the bit rate of a digital signal produced by any one of these multiplexers is 
slightly higher than the prescribed multiple of the incoming bit rate because of bit stuffing 
built into the design of each multiplexer; bit stuffing is discussed in the sequel. 

Moreover, it is important to recognize that the functions of a digital transmission 
facility is merely to carry a bit stream without interpreting what the bits themselves mean. 
However, the digital switches at the two ends of the facility do have a common under- 
standing of how to interpret the bits within the stream, such as whether the bits represent 
voice or data, framing format, signaling format, and so on. 

There are some basic problems involved in the design of a digital multiplexer, irre- 
spective of its grouping: 


1. Digital signals cannot be directly interleaved into a format that allows for their even- 
tual separation unless their bit rates are locked to a common clock. Rather, provision 
has to be made for synchronization of the incoming digital signals, so that they can 
be properly interleaved. 

2. The multiplexed signal must include some form of framing so that its individual 
components can be identified at the receiver. 

3. The multiplexer has to handle small variations in the bit rates of the incoming digital 
signals. For example, a 1000-km coaxial cable carrying 3 X 10° pulses per second 
will have about one million pulses in transit, with each pulse occupying about one 
meter of the cable. A 0.01 percent variation in the propagation delay, produced by 
a 1°F decrease in temperature, will result in 100 fewer pulses in the cable. Clearly, 
these pulses must be absorbed by the multiplexer. 


To tailor the requirements of synchronization and rate adjustment to accommodate small 
variations in the input data rates, we may use a technique known as bit stuffing. The idea 
here is to have the outgoing bit rate of the multiplexer slightly higher than the sum of the 
maximum expected bit rates of the input channels by stuffing in additional non-informa- 
tion carrying pulses. All incoming digital signals are stuffed with a number of bits sufficient 
to raise each of their bit rates to equal that of a locally generated clock. To accomplish bit 
stuffing, each incoming digital signal or bit stream is fed into an elastic store at the mul- 
tiplexer. The elastic store is a device that stores a bit stream in such a manner that the 
stream may be read out at a rate different from the rate at which it is read in. At the 
demultiplexer, the stuffed bits must obviously be removed from the multiplexed signal. 
This requires a method that can be used to identify the stuffed bits. To illustrate one such 
method, and also show one method of providing frame synchronization, we describe the 
signal format of the AT&T M12 multiplexer, which is designed to combine four DS1 bit 
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streams into one DS2 bit stream. This is the second level of the digital hierarchy discusseq 
earlier. 


P EXAMPLE 3.3 Signal Format of the AT&T M12 Multiplexer 


Figure 3.21 illustrates the signal format of the M12 multiplexer. Each frame is subdivided 
into four subframes. The first subframe (first line in Figure 3.21) is transmitted, then the 
second, the third, and the fourth, in that order. 

Bit-by-bit interleaving of the incoming four DS1 bit streams is used to accumulate a 
total of 48 bits, 12 from each input. A control bit is then inserted by the multiplexer. Each 
frame contains a total of 24 control bits, separated by sequences of 48 data bits. Three types 
of control bits are used in the M12 multiplexer to provide synchronization and frame indi- 
cation, and to identify which of the four input signals has been stuffed. These control bits are 
labeled F, M, and C in Figure 3.21. Their functions are as follows: 


1. The F-control bits, two per subframe, constitute the main framing pulses. The subscripts 
on the F-control bits denote the actual bit (0 or 1) transmitted. Thus the main framing 
sequence is FoF FoF FoFiFoF: or 01010101. 

2. The M-control bits, one per subframe, form secondary framing pulses to identify the 
four subframes. Here again the subscripts on the M-control bits denote the actual bit 
(0 or 1) transmitted. Thus the secondary framing sequence is MoM:ıM,M; or 0111. 

3. The C-control bits, three per subframe, are stuffing indicators. In particular, Cr refers 
to input channel I, Cu refers to input channel I, and so forth. For example, the three 
C-control bits in the first subframe following Mo in the first subframe are stuffing in- 
dicators for the first DS1 bit stream. The insertion of a stuffed bit in this DS1 bit stream 
is indicated by setting all three C-control bits to 1. To indicate no stuffing, all three are 
set to 0. If the three C-control bits indicate stuffing, the stuffed bit is located in the 
position of the first information bit associated with the first DS1 bit stream that follows 
the F,-control bit in the same subframe. In a similar way, the second, third, and fourth . 
DS1 bit streams may be stuffed, as required. By using majority logic decoding in the © 
receiver, a single error in any of the three C-control bits can be detected. This form of 
decoding means simply that the majority of the C-control bits determine whether an 
all-one or all-zero sequence was transmitted. Thus three 1s or combinations of two 1s 
and a 0 indicate that a stuffed bit is present in the information sequence, following the 
control bit F, in the pertinent subframe. On the other hand, three Os or combinations 
of two 0s and a 1 indicate that no stuffing is used. 


The demultiplexer at the receiving M12 unit first searches for the main framing sequence 

FoF FoF FoF: FoF. This establishes identity for the four input DS1 bit streams and also for the 

M- and C-control bits. From the MoMıMıM;ı sequence, the correct framing of the C-control 

bits is verified. Finally, the four DS1 bit streams are properly demultiplexed and destuffed. - 
The signal format described above has two safeguards: 


1. It is possible, although unlikely, that with just the FoF, FoF, FoFi FoF; sequence, one of 
the incoming DS1 bit streams may contain a similar sequence. This could then cause 


Mo (48) c (48) Fo (48) CG 4B] G [48] F; = (481 
M, (48) Cu (481 Fo (481 Cu #8 Cu (48) F} [481 
M 148) Cu 48h Fo U8 Cm M481 Cm [48] F, (48) 
Mm Me cy Ñ8l Fy “8 cy “8 Cw t48] Fry 148] 


Subframe First Frame Second Third Frame Stuffed 
markers stuffing markers stuffing stuffing markers bits 
indicators indicators indicators 


FıcuRE 3.21 Signal format of AT&T M12 multiplexer. 
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the receiver to lock onto the wrong sequence. The presence of the Mj>M,M,M, sequence 
provides verification of the genuine FF, FoF,FyF,FoF; sequence, thereby ensuring that 
the four DS1 bit streams are properly demultiplexed. 

2. The single-error correction capability built into the C-control bits ensures that the four 
DS1 bit streams are properly destuffed. 


The capacity of the M12 multiplexer to accommodate small variations in the input data 
rates can be calculated from the format of Figure 3.21. In each M frame, defined as the interval 
containing one cycle of Mj>M,M,M, bits, one bit can be stuffed into each of four input DS1 
bit streams. Each such signal has 12 X 6 X 4 = 288 positions in each M frame. Also, the T1 
bit stream has a bit rate equal to 1.544 Mb/s. Hence, each input can be incremented by 


1 
544 X 10? x — = 5, 
1.544 X 10 * Fea 5.4 kb/s 


This result is much larger than the expected change in the bit rate of the incoming DS1 bit 
stream, It follows therefore that the use of only one stuffed bit per input channel in each frame 
is sufficient to accommodate expected variations in the input signal rate. 

The local clock that determines the outgoing bit rate also determines the nominal stuffing 
rate S, defined as the average number of bits stuffed per channel in any frame. The 
M12 multiplexer is designed for S = 1/3. Accordingly, the nominal bit rate of the DS2 bit 
stream is 

49 288 
1.544 X 4x 48 x 388.5 6.312 Mb/s 
This also ensures that the nominal DS2 clock frequency is a multiple of 8 kHz (the nominal 
sampling rate of a voice signal), which is a desirable feature. 


3.11 Virtues, Limitations, 
and Modifications of PCM 


In a generic sense, pulse-code modulation (PCM) has emerged as the most favored mod- 
ulation scheme for the transmission of analog information-bearing signals such as voice 
and video signals. The advantages of PCM may all be traced to the use of coded pulses 
for the digital representation of analog signals, a feature that distinguishes it from all other 
analog methods of modulation. We may summarize the important advantages of PCM as 
follows: 


1. Robustness to channel noise and interference. 

2. Efficient regeneration of the coded signal along the transmission path. 

3. Efficient exchange of increased channel bandwidth for improved signal-to-noise ra- 
tio, obeying an exponential law. 

4. A uniform format for the transmission of different kinds of baseband signals, hence 
their integration with other forms of digital data in a common network. 

5. Comparative ease with which message sources may be dropped or reinserted in a 
time-division multiplex system. 

6. Secure communication through the use of special modulation schemes or encryption; 
the encryption and decryption of data are discussed in Appendix 5. 


These advantages, however, are attained at the cost of increased system complexity and 
increased channel bandwidth. These two issues are considered in the sequel in turn. 
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Although the use of PCM involves many complex operations, today they can all he 
implemented in a cost-effective fashion using commercially available and/or custom-made 
very-large-scale integrated (VLSI) chips. In other words, the requisite device technology 
for the implementation of a PCM system is already in place. Moreover, with continuin 
improvements in VLSI technology, we are likely to see an ever-expanding use of PCM fo, 
the digital transmission of analog signals. 

If, however, the simplicity of implementation is a necessary requirement, then we 
may use delta modulation as an alternative to pulse-code modulation. In delta modulation, 
the baseband signal is intentionally “oversampled” to permit the use of a simple quantizing 
strategy for constructing the encoded signal; delta modulation is discussed in Section 3,12, 

Turning next to the issue of bandwidth, we do recognize that the increased band. 
width requirement of PCM may have been a reason for justifiable concern in the past, 
Today, however, it is of no real concern for two different reasons. First, the increasing 
availability of wideband communication channels means that bandwidth is no longer a 
system constraint in the traditional way it used to be. Liberation from the bandwidth 
constraint has been made possible by the deployment of communication satellites for 
broadcasting and the ever-increasing use of fiber optics for networking; a discussion of 
these communication channel concepts was presented in the Background and Preview 
chapter. 

The second reason is that through the use of sophisticated data compression tech- 
niques, it is indeed possible to remove the redundancy inherently present in a PCM signal 
and thereby reduce the bit rate of the transmitted data without serious degradation in 
system performance. In effect, increased processing complexity (and therefore increased 
cost of implementation) is traded off for a reduced bit rate and therefore reduced band- 
width requirement. A major motivation for bit-rate reduction is for secure communication 
over radio channels that are inherently of low capacity. 


i 3.12 Delta Modulation 


In delta modulation® (DM), an incoming message signal is oversampled (i.e., at a rate 
much higher than the Nyquist rate) to purposely increase the correlation between adjacent 
samples of the signal. This is done to permit the use of a simple quantizing strategy for 
constructing the encoded signal. 

In its basic form, DM provides a staircase approximation to the oversampled version 
of the message signal, as illustrated in Figure 3.224. The difference between the input and 
the approximation is quantized into only two levels, namely, +A, corresponding to positive 
and negative differences. Thus if the approximation falls below the signal at any sampling 
epoch, it is increased by A. If on the other hand, the approximation lies above the signal, 
it is diminished by A. Provided that the signal does not change too rapidly from sample 
to sample, we find that the staircase approximation remains within +A of the input signal. 

Let m(t) denote the input (message) signal, and m(t) denote its staircase approxi- 
mation. For convenience of presentation, we adopt the following notation that is com- 
monly used in the digital signal processing literature: 


min] = m(nT), n=0, +1, +2,... 


where T, is the sampling period and m(nT.) is a sample of the signal m(t) taken at time 
t = nT, and likewise for the samples of other continuous-time signals. We may thea 
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FIGURE 3.22 Illustration of delta modulation. 


formalize the basic principles of delta modulation in the following set of discrete-time 
relations: 


ela] = m[n] — m,[n — 1] (3.52) 
e; = A sgn(e[n]) (3.53) 
mln] = m,[n — 1] + e,[n] (3.54) 


where e[7] is an error signal representing the difference between the present sample 1[7] 
of the input signal and the latest approximation m,[n — 1] to it, e,[n] is the quantized 
version of e[z], and sgn(:) is the signum function. Finally, the quantizer output m,[n] is 
coded to produce the DM signal. 

Figure 3.22q illustrates the way in which the staircase approximation m(t) follows 
variations in the input signal m(t) in accordance with Equations (3.52)-(3.54), and Figure 
3.22b displays the corresponding binary sequence at the delta modulator output. It is 
apparent that in a delta modulation system the rate of information transmission is simply 
equal to the sampling rate f, = 1/T,. 

The principal virtue of delta modulation is its simplicity. It may be generated by 
applying the sampled version of the incoming message signal to a modulator that involves 
a comparator, quantizer, and accumulator interconnected as shown in Figure 3.23a. The 
block labeled z~* inside the accumulator represents a unit delay, that is, a delay equal to 
one sampling period. (The variable z is commonly used in the z-transform, which is basic 
to the analysis of discrete-time signals and systems.) Details of the modulator follow di- 
rectly from Equations (3.52)—(3.54). The comparator computes the difference between its 
two inputs. The quantizer consists of a bard limiter with an input-output relation that is 
a scaled version of the signum function. The quantizer output is then applied to an accu- 
mulator, producing the result 


mn] = A > sgn(e[i]) 
m (3.55) 


= > eE] 
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FIGURE 3.23 DM system. (a) Transmitter. (b) Receiver. 


which is obtained by solving Equations (3.53) and (3.54) for m,[7]. Thus, at the sampling 
instant nT., the accumulator increments the approximation by a step A ina positive or 
negative direction, depending on the algebraic sign of the error sample e[n]. If the input 
sample [7] is greater than the most recent approximation m,|n], a positive increment 
+A is applied to the approximation. If, on the other hand, the input sample is smaller, a 
negative increment —A is applied to the approximation. In this way, the accumulator does 
the best it can to track the input samples by one step (of amplitude +A or —A) at a time. 
In the receiver shown in Figure 3.238, the staircase approximation 7,(¢) is reconstructed 
by passing the sequence of positive and negative pulses, produced at the decoder output, 
through an accumulator in a manner similar to that used in the transmitter. The out-of 
band quantization noise in the high-frequency staircase waveform m,(t) is rejected by 
passing it through a low-pass filter, as in Figure 3.23b, with a bandwidth equal to the 
original message bandwidth. 

Delta modulation is subject to two types of quantization error: slope overload dis- 
tortion and granular noise. We will discuss the case of slope overload distortion first. 

We observe that Equation (3.54) is the digital equivalent of integration in the sens¢ 
that it represents the accumulation of positive and negative increments of magnitude A. 
Also, denoting the quantization error by qlz], as shown by 


mn] = mln] + qin (3.56) 
we observe from Equation (3.52) that the input to the quantizer is 


eln] = min] — mln — 1] - ala — 1] (3.57) 


3.12 Delta Modulation 221 


Granular noise 


Slope-overload 
distortion 






mit) 


Staircase 
approximation 
mÀ 


FIGURE 3.24 Illustration of the two different forms of quantization error in delta modulation. 


Thus except for the quantization error g[# — 1], the quantizer input is a first backward 
difference of the input signal, which may be viewed as a digital approximation to the 
derivative of the input signal or, equivalently, as the inverse of the digital integration 
process. If we consider the maximum slope of the original input waveform m(t), it is 
clear that in order for the sequence of samples {v,[7]} to increase as fast as the input 
sequence of samples {m[n]} in a region of maximum slope of m(t), we require that the 
condition 


A 
T: 


dm(t) 


di (3.58) 


= max 








be satisfied. Otherwise, we find that the step-size A is too small for the staircase approxi- 
mation m,(t) to follow a steep segment of the input waveform m(t), with the result that 
m,(t) falls behind a(t), as illustrated in Figure 3.24. This condition is called slope overload, 
and the resulting quantization error is called slope-overload distortion (noise). Note that 
since the maximum slope of the staircase approximation 7,(t) is fixed by the step size A, 
increases and decreases in ,(t) tend to occur along straight lines. For this reason, a delta 
modulator using a fixed step size is often referred to as a linear delta modulator. 

In contrast to slope-overload distortion, granular noise occurs when the step size A 
is too large relative to the local slope characteristics of the input waveform m(t), thereby 
causing the staircase approximation m,(t) to hunt around a relatively flat segment of the 
input waveform; this phenomenon is also illustrated in Figure 3.24. Granular noise is 
analogous to quantization noise in a PCM system. 

We thus see that there is a need to have a large step-size to accommodate a wide 
dynamic range, whereas a small step size is required for the accurate representation of 
relatively low-level signals. It is therefore clear that the choice of the optimum step size 
that minimizes the mean-square value of the quantization error in a linear delta modulator 
will be the result of a compromise between slope-overload distortion and granular noise. 
To satisfy such a requirement, we need to make the delta modulator “adaptive,” in the 
sense that the step size is made to vary in accordance with the input signal; this issue is 
discussed further in a computer experiment presented in Section 3.16. 


a DELTA-SIGMA MODULATION 


As mentioned earlier, the quantizer input in the conventional form of delta modulation 
may be viewed as an approximation to the derivative of the incoming message signal. This 
behavior leads to a drawback of delta modulation in that transmission disturbances such 
as noise result in an accumulative error in the demodulated signal. This drawback can be 
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overcome by integrating the message signal prior to delta modulation. The use of integra. 
tion in the manner described here has also the following beneficial effects: 


> The low-frequency content of the input signal is pre-emphasized. 

> Correlation between adjacent samples of the delta modulator input is increased 
which tends to improve overall system performance by reducing the variance of the 
error signal at the quantizer input. 

> Design of the receiver is simplified. 


A delta modulation scheme that incorporates integration at its input is called delta-sigma 
modulation (D-2M).? To be more precise, however, it should be called sigma-delta mod- 
ulation, because the integration is in fact performed before the delta modulation. Never. 
theless, the former terminology is the one commonly used in the literature. 

Figure 3.254 shows the block diagram of a delta-sigma modulation system. In this 
diagram, the message signal m(t) is defined in its continuous-time form, which means that 
the pulse modulator now consists of a hard-limiter followed by a multiplier; the latter 
component is also fed from an external pulse generator (clock) to produce a 1-bit encoded 
signal. The use of integration at the transmitter input clearly requires an inverse signal 
emphasis, namely, differentiation, at the receiver. The need for this differentiation is, how- 
ever, eliminated because of its cancellation by integration in the conventional DM receiver. 
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FIGURE 3.25 Two equivalent versions of delta-sigma modulation system. 
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Thus the receiver of a delta-sigma modulation system consists simply of a low-pass filter, 
as indicated in Figure 3.25a. 

Moreover, we note that integration is basically a linear operation. Accordingly, we 
may simplify the design of the transmitter by combining the two integrators 1 and 2 of 
Figure 3.252 into a single integrator placed after the comparator, as shown in Figure 3.25b. 
This latter form of the delta-sigma modulation system is not only simpler than that of 
Figure 3.25a, but it also provides an interesting interpretation of delta-sigma modulation 
as a “smoothed” version of 1-bit pulse-code modulation: The term smoothness refers to 
the fact that the comparator output is integrated prior to quantization, and the term 1-bit 
merely restates that the quantizer consists of a hard-limiter with only two representation 
levels. 

In delta modulation, simplicity of implementations of both the transmitter and re- 
ceiver is attained by using a sampling rate far in excess of that needed for pulse-code 
modulation. The price paid for this benefit is a corresponding increase in the transmission 
and therefore channel bandwidth. There are, however, applications where channel band- 
width is at a premium, in which case we have the opposite requirement to that in delta 
modulation. Specifically, we may wish to trade increased system complexity for a reduced 
channel bandwidth. A signal-processing operation basic to the attainment of this latter 
design objective is prediction, the linear form of which is discussed next. 


| 3.13 Linear Prediction 


Consider a finite-duration impulse response (FIR) discrete-time filter configured as in Fig- 
ure 3.26, which involves the use of three functional blocks: 


1. Set of p unit-delay elements, each of which is represented by z7'. 


2. Set of multipliers involving the filter coefficients w4, w2,..., Wp. 


3. Set of “adders” used to sum the scaled versions of the delayed inputs x[x — 1], 
x[n — 2],..., x[a — p] to produce the output <[7]. The filter output [n] or more 
precisely, the linear prediction of the input, is thus defined by the convolution sum 


[x] = S w,x|[n — k] (3.59) 
k=1 


where p, the number of unit-delay elements, is called the prediction order. 


The actual sample at time #T, is x[n]. The prediction error, denoted by e[n], is defined 
as the difference between x[] and the prediction £[7], as shown by 


ela] = x[nx] — ŝia] (3.60) 
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FIGURE 3.26 Block diagram of a linear prediction filter of order p. 
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The design objective is to choose the filter coefficients w1, W2,..., Wp SO as tO Minimize 
an index of performance, J, defined as the mean-square error: 


J = Ee |] (3.61) 


Substituting Equations (3.59) and (3.60) into (3.61) and then expanding terms, we may 
reformulate the index of performance as 


P PP 
J = E[x*[n]] - 2 2, w,E{x[n)x[n — kl] + 2 2 ww,E|x[n — jle[e — kI] (3.62) 


We assume that the input signal x(t) is the sample function of a stationary process 
X(t) of zero mean; that is, E[x[7]] is zero for all n. Define 
o% = variance of a sample of the process X(t) at time #T, 
= Efir] — (Elx[nl])? 
= Efx?[a]] 
Rx(kT.) = autocorrelation of the process X(t) for a lag of kT, 
= Rx{k] 
= E[x[n]x[n — k]] 


Accordingly, we may rewrite Equation (3.62) in the simplified form 


P A 
J=0%-2 2 w,Rx{k] + > 2 ww Rxlk- j] (3.63) 
= Ae 


Hence differentiating the index of performance J with respect to the filter coefficient wy 
setting the result equal to zero, and then rearranging terms, we obtain 


b 
> wRylk — j| = Rx[k] = Rxl-k] k =1,2,...3P (3.64) 


The optimality equations (3.64) are called the Wiener-Hopf equations for linear prediction. 
We find it convenient to reformulate the Wiener-Hopf equations (3.64) in matrix 
form. Let 


w, = p-by-1 optimum coefficient vector 
= T 
= [w W562 +9 wp] 

rx = p-by-1 autocorrelation vector 


= [Rx[1], Rx[2], .- - > Rxfel]” 
~ Ry = p-by-p autocorrelation matrix 


R,{0] R,{1] --» Rsip - 1] 
O| RAI Rel] o Ral = 21 
Rylp - 1] Rep aa -R 


We may thus simplify the set of equations (3.64) as 
Ryw, = Ex (3.65) 
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We assume that the autocorrelation matrix Ry is nonsingular, so that its inverse exists. 
We may then solve Equation (3.65) for the coefficient vector w, by multiplying both sides 
of this equation by the inverse matrix Rx', obtaining the optimum solution 


ow, = Rx'ry (3.66) 


Note that all the elements on the main diagonal of the autocorrelation matrix Rx 
are equal to Rx[0] = o%, and the elements on any other diagonal parallel to the main 
diagonal are also equal. A square matrix having this property is said to be Toeplitz, which 
is a direct consequence of the assumption that the input signal x(t) is drawn from a sta- 
tionary process. The practical significance of the Toeplitz property is that the correlation 
matrix Rx is uniquely defined by the set of autocorrelation values Rx[0], Rx[1],..., 
Rx[p — 1]. The autocorrelation vector rx is defined by the set of autocorrelation values 
Rx{1], Rx{2],..., Rx[p]. It follows therefore that the p filter coefficients of the linear 
optimum predictor are uniquely defined by the variance o4 = Rx[0] and p values of the 
autocorrelation function of the process X(t) for lags of T., 2T., ... > pT;. 

The minimum mean-square value of the prediction error is obtained by substituting 
Equation (3.64) into (3.63), which yields (after simplification) 


Jain = O% — TXRX rx (3.67) 


The quadratic term rxRx'rx is always positive. Accordingly, the mean-square error J nin Of 
the optimum linear predictor defined by Equation (3.67) is always less than the variance 
o% of the input sample that is being predicted. 


© LINEAR ADAPTIVE PREDICTION 


The use of Equation (3.66) for calculating the weight vector of a linear predictor requires 
knowledge of the autocorrelation function Rx[k] of the input sequence {x[7]} for lags 
k =0,1,..., p, where p is the prediction order. What if knowledge of Rx[k] for varying 
k is not available? In these situations, which occur frequently in practice, we may resort 
to the use of an adaptive predictor. 

The predictor is adaptive in the following sense: 


> Computation of the tap weights w,, k = 1, 2,..., p, proceeds in a “recursive” 
manner, starting from some arbitrary initial values of the tap weights. 

> The algorithm used to adjust the tap weights (from one iteration to the next) is “self- 
designed,” operating solely on the basis of available data. 


The aim of the algorithm is to find the minimum point of the bowl-shaped error surface 
that describes the dependence of the cost function J on the tap weights. It is therefore 
intuitively reasonable that successive adjustments to the tap-weights of the predictor be 
made in the direction of the steepest descent of the error surface, that is, in a direction 
opposite to the gradient vector whose elements are defined by 


aag FLR sp (3.68) 


This is indeed the idea behind the method of steepest descent. Let w,[n] denote the value 
of the kth tap-weight at iteration n. Then the updated value of this weight at iteration 
n + 1 is defined by 


1 
wale + 1] = wale] — Fuge k= 1,2,...,7 (3.69) 
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where u is a step-size parameter that controls the speed of adaptation, and the factor 1/2 
is included for convenience of presentation. Differentiating the cost function J of Equation 
(3.63) with respect to wg, we readily find that 


ll 


P? 
gr = —2Rx[k] + 2 > w;Rxlk — jl 


(3.70) 


P 
—2Elx|n]x[n — k]] + 2 > w E[x[n — jlx[n — kl, kR=1,2,...,p 
j=l 


This formula for g, could do with further simplification, which is achieved by using iy. 
stantaneous values as estimates of the autocorrelation functions Rx[k] and Rx[k — jl. That 
is, we ignore the expectation operators in Equation (3.70) to facilitate the adaptive process 
on a step-by-step basis. We may thus express the corresponding estimate of g, at iteration 
nas 


2 
ĝln] = —2x[n|x[n — k] + 2 > w,{n|x[n — jlxla-k], k=1,2,...3 p (3.71) 
= 
Note that for an input x[n] drawn from a stationary process the gradient g, is a determin- 
istic quantity, whereas the estimate g,[7] is the sample value of a random variable. 


In any event, substituting Equation (3.71) into (3.69) and factoring the common 
term x[n — k], we may write 


w,[n + 1] 


p 
w,[n] + xin — a(x - > ôni” - i) 

j= (3.72) 
wln] + px[n a= Rle(n], k=1,2,...57 


where e[n] is the prediction error defined as 


p 
eln] = x[n] — È bnl” = j (3.73) 
=Z 
In Equations (3.72) and (3.73), we have used 1, as an estimate of the kth tap-weight to 
distinguish it from the actual value w,. Note also that x[7] plays the role of a “desired 
response” for computing the recursive adjustments applied to the tap-weights of the 
predictor. 

Equations (3.72) and (3.73) constitute the popular least-mean-square (LMS) algo- 
rithm for linear adaptive prediction, the operation of which is depicted in Figure 3.27. The 
reason for popularity of this adaptive filtering algorithm is the simplicity of its implemen- 
tation. In particular, the computational complexity of the algorithm, measured in terms 
of the number of additions and multiplications, is linear in the prediction order p. 
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FIGURE 3.27 Block diagram illustrating the linear adaptive prediction process. 
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The LMS algorithm is a stochastic adaptive filtering algorithm, stochastic in the sense 
that, starting from the initial condition defined by {w,[0]}é-1, it seeks to find the minimum 
point of the error surface by following a zig-zag path. Moreover, it never finds this mini- 
mum point exactly. Rather, it executes a random motion around the minimum point of 
the error surface, once steady-state conditions are established. 

With this material on linear prediction at hand, we are ready to discuss practical 
improvements on the performance of pulse-code modulation. 


j 3.14 Differential Pulse-Code Modulation 


When a voice or video signal is sampled at a rate slightly higher than the Nyquist rate as 
usually done in pulse-code modulation, the resulting sampled signal is found to exhibit a 
high degree of correlation between adjacent samples. The meaning of this high correlation 
is that, in an average sense, the signal does not change rapidly from one sample to the 
next, and as a result, the difference between adjacent samples has a variance that is smaller 
than the variance of the signal itself. When these highly correlated samples are encoded, 
as in the standard PCM system, the resulting encoded signal contains redundant infor- 
mation, This means that symbols that are not absolutely essential to the transmission of 
information are generated as a result of the encoding process. By removing this redundancy 
before encoding, we obtain a more efficient coded signal, which is the basic idea behind 
differential pulse-code modulation. 

Now if we know the past behavior of a signal up to a certain point in time, we may 
use prediction to make an estimate of a future value of the signal as described in Section 
3.13. Suppose then a baseband signal m(t) is sampled at the rate fs = 1/T, to produce the 
sequence {m[]} whose samples are T, seconds apart. The fact that it is possible to predict 
future values of the signal m(t) provides motivation for the differential quantization scheme 
shown in Figure 3.28a. In this scheme, the input signal to the quantizer is defined by 


ejn] = m[n] — m[n] (3.74) 


which is the difference between the unquantized input sample [7] and a prediction of it, 
denoted by #[7]. This predicted value is produced by using a linear prediction filter whose 
input, as we will see, consists of a quantized version of the input sample [n]. The differ- 
ence signal e[7] is the prediction error, since it is the amount by which the prediction filter 
fails to predict the input exactly. By encoding the quantizer output, as in Figure 3.284, we 
obtain a variant of PCM known as differential pulse-code modulation” (DPCM). 

The quantizer output may be expressed as 


ealn] = ela] + qin] (3.75) 


where qf] is the quantization error: According to Figure 3.282, the quantizer output e,[7] 
is added to the prédicted value #[71] to produce the prediction-filter input 


mln] = mln] + e,[n] (3.76) 
Substituting Equation (3.75) into (3.76), we get 
man] = la] + efa] + qla] - (3.77) 


However, from Equation (3.74) we observe that the sum term m[n] + efr] is equal to the 
input sample m|[n]. Therefore, we may simplify Equation (3.77) as 


mala] = mia] + qia] (3.78) 
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FIGURE 3.28 DPCM system. (a) Transmitter. (b) Receiver. 


which represents a quantized version of the input sample m[n]. That is, irrespective of the 
properties of the prediction filter, the quantized sample m,[7] at the prediction filter input 
differs from the original input sample m[7] by the quantization error qin]. Accordingly, if 
the prediction is good, the variance of the prediction error e|7] will be smaller than the 
variance of mn], so that a quantizer with a given number of levels can be adjusted to 
produce a quantization error with a smaller variance than would be possible if the input 
sample m[1] were quantized directly as in a standard PCM system. 

The receiver for reconstructing the quantized version of the input is shown in Figure 
3.28b. It consists of a decoder to reconstruct the quantized error signal. The quantized 
version of the original input is reconstructed from the decoder output using the same 
prediction filter used in the transmitter of Figure 3.28a. In the absence of channel noise, 
we find that the encoded signal at the receiver input is identical to the encoded signal at 
the transmitter output. Accordingly, the corresponding receiver output is equal to ml”), 
which differs from the original input m/[n] only by the quantization error qin] incurred as 
a result of quantizing the prediction error e[n]. 

From the foregoing analysis we observe that, in a noise-free environment, the pre 
diction filters in the transmitter and receiver operate on the same sequence of samples, 
m,{n]. It is with this purpose in mind that a feedback path is added to the quantizer in the 
transmitter, as shown in Figure 3.282. 

Differential pulse-code modulation includes delta modulation as a special case. In 
particular, comparing the DPCM system of Figure 3.28 with the DM system of Figu? 
3.23, we see that they are basically similar, except for two important differences: the ust 
of a one-bit (two-level) quantizer in the delta modulator and the replacement of the pre 
diction filter by a single delay element (i.e., zero prediction order). Simply put, DM is the 
4-bit version of DPCM. Note that unlike a standard PCM system, the transmitters of both 
the DPCM and DM involve the use of feedback. 

DPCM, like DM, is subject to slope-overload distortion whenever the input signal 
changes too rapidly for the prediction filter to track it. Also, like PCM, DPCM suffers 
from quantization noise. 
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a PROCESSING GAIN 


The output signal-to-noise ratio of the DPCM system shown in Figure 3.28 is, by 
definition, . 


TM 


(SNR)o = ah 


(3.79) 
where o% is the variance of the original input sample [7], assumed to be of zero mean, 
and o@ is the variance of the quantization error q[#]. We may rewrite Equation (3.79) as 
the product of two factors as follows: 


AE. 

SNR)o = (2) 
ii (a o (3.80) 

= G,(SNR)o 


where o7 is the variance of the prediction error. The factor (SNR)g is the signal-to- 
quantization noise ratio, which is defined by 
2 
(SNR)g = 22 (3.81) 
TQ 
The other factor G, is the processing gain produced by the differential quantization 
scheme; it is defined by 


oe 
G= 


The quantity G,, when greater than unity, represents a gain in signal-to-noise ratio that 
is due to the differential quantization scheme of Figure 3.28. Now, for a given baseband 
(message) signal, the variance oj, is fixed, so that G, is maximized by minimizing the 
variance of of the prediction error e[n]. Accordingly, our objective should be to design 
the prediction filter so as to minimize o3. 

In the case of voice signals, it is found that the optimum signal-to-quantization noise 
advantage of DPCM over standard PCM is in the neighborhood of 4 to 11 dB. The greatest 
improvement occurs in going from no prediction to first-order prediction, with some ad- 
ditional gain resulting from increasing the order of the prediction filter up to 4 or 5, after 
which little additional gain is obtained. Since 6 dB of quantization noise is equivalent to 
1 bit per sample by virtue of Equation (3.35), the advantage of DPCM may also be ex- 
pressed in terms of bit rate. For a constant signal-to-quantization noise ratio, and assuming 
a sampling rate of 8 kHz, the use of DPCM may provide a saving of about 8 to 16 kb/s 
{i.e., 1 to 2 bits per sample) compared to the standard PCM. 


3.15 Adaptive Differential 
Pulse-Code Modulation 


The use of PCM for speech coding at the standard rate of 64 kb/s demands a high channel 
bandwidth for its transmission. In certain applications, however, such as secure transmis- 
sion over radio channels that are inherently of low capacity, channel bandwidth is at a 
premium. In applications of this kind, there is a definite need for speech coding at low bit 
rates, while maintaining acceptable fidelity or quality of reproduction. 


(3.82) 
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For coding speech at low bit rates, a waveform coder of prescribed configuration is 
optimized by exploiting both statistical characterization of speech waveforms and Drop. 
erties of hearing. In particular, the design philosophy has two aims in mind: 


1. To remove redundancies from the speech signal as far as possible. 


2. To assign the available bits to code the nonredundant parts of the speech signal ing 
perceptually efficient manner. 


As we strive to reduce the bit rate from 64 kb/s (used in standard PCM) to 32, 16, 8, ang 
4 kb/s, the schemes used for redundancy removal and bit assignment become increasingly 
more sophisticated. As a rule of thumb, in the 64 to 8 kb/s range, the computational 
complexity (measured in terms of multiply-add operations) required to code speech in. 
creases by an order of magnitude when the bit rate is halved, for approximately equa 
speech quality. 

In this section, we describe adaptive differential pulse-code modulation (ADPCM), 
which permits the coding of speech at 32 kb/s through the combined use of adaptive 
quantization and adaptive prediction; the number of eight bits per sample required in the 
standard PCM is thereby reduced to four. The term adaptive used herein means being 
responsive to changing level and spectrum of the input speech signal. The variation of 
performance with speakers and speech material, together with variations in signal level 
inherent in the speech communication process, make the combined use of adaptive quan. 
tization and adaptive prediction necessary to achieve best performance over a wide range 
of speakers and speaking situations. 

Adaptive quantization refers to a quantizer that operates with a time-varying step- 
size A[n]. At any given sampling instant identified by the index 7, the adaptive quantizer 
is assumed to have a uniform transfer characteristic. The step-size A[n] is varied so as to 
match the variance o% of the input sample m[n]. In particular, we write 


Aln] = b6u[n] (3.83) 


where ¢ is a constant, and &y[7] is an estimate of the standard deviation op,[11] (i.e., square 
root of the variance o4). For a nonstationary input, oj[7] is time varying. The problem 
of adaptive quantization according to Equation (3.83) is, therefore, one of computing the 
estimate 6,[”] continuously. 

The implementation of Equation (3.83) may proceed in one of two ways: 


1. Adaptive quantization with forward estimation (AQF), in which unquantized sam- 
ples of the input signal are used to derive forward estimates of ol"). 

2. Adaptive quantization with backward estimation (AQB), in which samples of the 
quantizer output are used to derive backward estimates of o4[7]. 


The AQF scheme requires the use of a buffer to store unquantized samples of the input 
speech signal needed for the learning period. It also requires the explicit transmission of 
level information (typically, about 5 to 6 bits per step-size sample) to a remote decoder, 
thereby burdening the system with additional side information that has to be transmitted 
to the receiver. Moreover, a processing delay (on the order of 16 ms for speech) in the 
encoding operation results from the use of AQF, which is unacceptable in some applica 
tions. The problems of level transmission, buffering, and delay intrinsic to AQF are a 
avoided in AQB. In the latter scheme, the recent history of the quantizer output is used to 
extract information for the computation of the step size A[7]. In practice, AQB is therefore 
usually preferred over AQF. f 
Figure 3.29 shows the block diagram of an adaptive quantizer with backward est- 
mation. It represents a nonlinear feedback system; hence, it is not obvious that the syste 
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FIGURE 3.29 Adaptive quantization with backward estimation (AQB). 


will be stable. However, if the quantizer input m[n] is bounded, then the backward estimate 
Gln] and the corresponding step size A[7] are as well; under such a condition, the system 
is indeed stable. 

The use of adaptive prediction in ADPCM is justified because speech signals are 
inherently nonstationary, a phenomenon that manifests itself in the fact that the autocor- 
relation function and power spectral density of speech signals are time-varying functions 
of their respective arguments. This implies that the design of predictors for such inputs 
should likewise be time varying, that is, adaptive. As with adaptive quantization, there are 
two schemes for performing adaptive prediction: 


1. Adaptive prediction with forward estimation (APF), in which unquantized samples 
of the input signal are used to derive estimates of the predictor coefficients. 

2. Adaptive prediction with backward estimation (APB), in which samples of the quan- 
tizer output and the prediction error are used to derive estimates of the predictor 
coefficients. 


However, APF suffers from the same intrinsic disadvantages (side information, buffering, 
and delay) as AQF. These disadvantages are eliminated by using the APB scheme shown 
in Figure 3.30, where the box labeled “logic for adaptive prediction” represents the al- 
gorithm for updating the predictor coefficients. In the latter scheme, the optimum predictor 
coefficients are estimated on the basis of quantized and transmitted data; they can therefore 
be updated as frequently as desired, say, from sample to sample. Accordingly, APB is the 
preferred method of prediction for ADPCM. 

The LMS algorithm for the predictor, described in Section 3.13, and an adaptive 
scheme for the quantizer, based on Equation (3.83), have been combined in a synchronous 
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FIGURE 3.30 Adaptive prediction with backward estimation (APB). 
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fashion for the design of both the encoder and decoder. The performance of this combi. 
nation is so impressive at 32 kb/s that ADPCM is now accepted internationally as a stan. 
dard coding technique for voice signals, along with 64 kb/s using standard PCM. 


3.16 Computer Experiment: 
Adaptive Delta Modulation 
A simple form of AQB is to be found in the modification of linear delta modulation (LDM) 


to form adaptive delta modulation (ADM). The principle underlying all ADM algorithms 
is two-fold: 


1. If successive errors are of opposite polarity, then the delta modulator is operating in 
its granular mode; in this case, it may be advantageous to reduce the step size. 

2. If, however, successive errors are of the same polarity, then the delta modulator 3s 
operating in its slope-overload mode; in this second case, the step size should be 
increased. 


Thus by varying the step-size in accordance with this principle, the delta modulator is 
enabled to cope with changes in the input signal. 

Figure 3.31 shows the block diagram of an ADM based on increasing or decreasing 
the step size by a factor of 50 percent at each iteration of the adaptive process. The al- 
gorithm for adaptation of the step size is defined by” 


lAl- 4]| if > 
Aln] = i (m{n] + 0.5m,[" — 1}) if A[n — 1] = Amin (3.84) 


Amin if Ala — 1] < Amin 


where Af7ı] is the step size at iteration (time step) ⁄ of the algorithm, and m,|n] is the 1-bit 
quantizer output that equals +1. 

In this experiment we use a sinusoidal input signal to demonstrate the reconstruction 
performance of the ADM algorithm based on Equation (3.84), and compare it to the 
performance of a corresponding linear delta modulator (LDM). Details of the experiment 
are as follows: 


Input signal: 
m(t) = A sin(27fut) 


where amplitude A = 10, frequency fn = f,/100, and f; = sampling frequency. 


Linear delta modulation (LDM): 
Step size A{n] = 1 for all # 


Adaptive delta modulation (ADM): 
1 
Amin = = 
min 8 
The results of the experiment are plotted in Figure 3,32. Part a of Figure 3.32 is fot 


LDM, and part of the figure is for ADM. From the waveforms presented here, we may 
make the following observations: 


» ADM tracks changes in the sinusoidal input signal much better than LDM. This 
improvement in the performance of ADM is due to adaptation of the step size 0 
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FIGURE 3.31 Adaptive delta modulation system: (a) Transmitter. (b) Receiver. 
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successive iterations of the algorithm. In particular, the reduced step size of the ADM 
results in smaller quantization errors near the extremities of the input signal than the 
LDM. However, both modulation schemes produce comparable quantization errors 


in regions of the input signal where the slope is moderately high. 


> The improved tracking performance of the ADM results in an output signal with a 


much lower bit rate, on the average, than the LDM. 
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FIGURE 3.32 Waveforms resulting from the computer experiment on delta modulation: 
(a) Linear delta modulation. (b) Adaptive delta modulation. 
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i 3. 17 MPEG Audio Coding Standard 


Speech (voice) and audio signals are similar in that, in both cases, the quality of a coding 
scheme is based on the properties of human auditory perception. In the case of speech 
signals, we have efficient coding schemes (e.g., ADPCM) because a speech production 
model is available. Unfortunately, nothing similar exists for audio signals. 

In this section, we revisit the MPEG-1/audio coding standard briefly described in the 
Background and Preview chapter; MPEG stands for Motion Picture Experts Group, and 
the suffix 1 is intended to mean it is the first in a series of several standards. *? Like ADPCM 
the MPEG-1/audio coding standard is a lossy compression system, but it differs from 
ADPCM in an important practical respect: The MPEG-1 standard is capable of achieving 
transparent, perceptually lossless compression of stereophonic audio signals at high 
sampling rates. In particular, subjective listening tests performed by the MPEG/audio 
committee, under very difficult listening conditions, have shown that even with a 6-to-1 
compression ratio, the coded and original audio signals are perceptually indistinguishable, 

The MPEG-1/audio coding standard achieves this remarkable performance by ex. 
ploiting two psychoacoustic characteristics of the human auditory system: 


1. Critical bands. 

The inner ear'* of the auditory system represents the power spectra of incoming signals 
on a nonlinear scale in the form of limited frequency bands called the critical bands. The 
audible frequency band, extending up to 20 kHz, is covered by 25 critical bands, whose 
individual bandwidths increase with frequency. Loosely speaking, the auditory system may 
be modeled as a band-pass filter bank, consisting of 2.5 overlapping band-pass filters with 
bandwidths less than 100 Hz for the lowest audible frequencies and up to 5 kHz for the 
highest audible frequencies. 


2. Auditory masking. 

Auditory masking or noise masking is a frequency-domain phenomenon that arises when 
a low-level signal (the maskee) and a high-level signal (the masker) occur simultaneously 
and are close enough to each other in frequency. If the low-level signal lies below a masking 
threshold, it is made inaudible (i.e., masked) by the stronger signal. The auditory-masking 
phenomenon is most pronounced when both signals lie in the same critical band, and less 
effective when they lie in neighboring bands. 


Figure 3.33 illustrates the definition of masking threshold and related parameters for 
a pair of adjacent frequency bands; it is assumed that the masker (i.e., the high-level signal} 
lies inside. the dark-shaded critical band. The low-level signals lying inside this dark area 
and below the masking threshold are masked by the stronger signal. From Figure 3.33 we 
see that the masking threshold varies with frequency across the critical band. Accordingly, 
we may define a minimum masking threshold for a critical band, below which all low- 
level signals that lie inside that band are made inaudible by the stronger signal. The power 
difference, expressed in decibels, between the masker and the minimum masking threshold, 
is termed the signal-to-mask ratio (SMR). Figure 3.33 also includes the signal-to-noise 
ratio (SNR) for an R-bit quantizer. The difference between SMR and SNR is the noise-t0- 
mask ratio (NMR) for an R-bit quantizer as shown by 


NMR = SMR — SNR (3.85) 


where all three terms are expressed in dBs. Within a critical band, the quantization no 
is inaudible as long as the NMR for the pertinent quantizer is negative. 
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FIGURE 3.33 Illustrating the definitions of masking threshold and related parameters. The high- 
level signal (masker) lies inside the darker-shaded critical band, hence the masking is more 
effective in this band than in the neighboring band shown in lighter shading. (Adapted from Noll 
(1998) with permission of the CRC Press.) 


With this background on the psychoacoustics of the auditory system, we are now 
ready to describe the operation of the MPEG-1/audio coding standard. Figure 3.34 shows 
the basic block diagrams of the encoder and decoder. The encoder consists of four func- 
tional units: time-to-frequency mapping network, psychoacoustic model, quantizer and 
coder, and frame-packing unit. The decoder consists of three functional units: frame- 
unpacking unit, frequency-sample reconstruction network, and frequency-to-time map- 
ping network. The psychoacoustic model is thus only necessary in the encoder. 

Starting with a description of the encoder first, the function of the time-to-frequency 
mapping network is to decompose the input audio signal into multiple subbands for cod- 
ing. The mapping is performed in three layers, labeled I, I, and II, which are of increasing 
complexity, delay, and subjective perceptual performance. The algorithm in layer I üsés a 
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FIGURE 3.34 MPEG/Audio coding system. (a) Transmitter. (b) Receiver. 
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band-pass filter bank that divides the audio signal into 32 constant-width subbands; this 
filter bank is also found in layers II and I. In light of our previous remarks on the non. 
uniformly spaced critical bands, the design of this filter bank is a compromise between 
computational efficiency and perceptual performance. The algorithm in layer I is a simple 
enhancement of layer I; it improves the compression performance by coding the data iy 
larger groups. Finally, the layer I algorithm is much more refined in that it is designeg 
to achieve frequency resolutions closer to the partitions between the critical bands. 

The psychoacoustic model is the key component in the encoder. Its function is to 
analyze the spectral content of the input audio signal and thereby compute the signal-to. 
mask ratio for each subband in each of the three layers. This information is, in turn, used 
by the quantizer-coder to decide how to apportion the available number of bits for the 
quantization of the subband signals. This dynamic allocation of bits is performed so as to 
minimize the audibility of the quantization noise. Finally, the frame-packing unit assembles 
the quantized audio samples into a decodable bit stream. 

The decoder simply reverses the signal-processing operations performed in the en- 
coder, converting the received stream of encoded bits into a time-domain audio signal, 

To sum up, the MPEG-1/audio coding standard represents the state of the art in the 
coding of audio signals. Layer I achieves a compression ratio of 4 at an approximate stereo 
bit rate of 384 kb/s for transparent quality of performance. The corresponding compres- 
sion ratios for layers H and. III are 8 and 12 at approximate stereo bit rates of 192 kb/s 
and 128 kb/s, respectively. The subjective quality of the MPEG-1/audio coding standard 
is equivalent to compact disc quality (16-bit PCM) for many types of music; the compact 
disc (CD) is today’s de facto standard of digital audio representation. f 


| 3.18 Summary and Discussion 


In this chapter we introduced two fundamental and complementary processes: 


> Sampling, which operates in the time domain; the sampling process is the link be- 
tween an analog waveform and its discrete-time representation. 

> Quantization, which operates in the amplitude domain; the quantization process is 
the link between an analog waveform and its discrete-amplitude representation. 


The sampling process builds on the sampling theorem, which states that a strictly band- 
limited signal with no frequency components higher than W Hz is represented uniquely 
by a sequence of samples taken at a uniform rate equal to or greater than the Nyquist rate 
of 2W samples per second. The quantization process exploits the fact that any human 
sense, as ultimate receiver, can only detect finite intensity differences. 

The sampling process is basic to the operation of all pulse modulation systems, which 
may be classified into analog pulse modulation and digital pulse modulation. The disti- 
guishing feature between them is that analog pulse modulation systems maintain a contin 
uous amplitude representation of the message signal, whereas digital pulse modulation 
systems also employ quantization to provide a representation of the message signal that is 
discrete in both time and amplitude. 

Analog pulse modulation results from varying some parameter of the transmitted. 
pulses, such as amplitude, duration, or position, in which case we speak of pulse-amplitude 
modulation (PAM), pulse-duration modulation (PDM), or pulse-position modulation 
(PPM), respectively. In time-division multiplexing (TDM) of several channels, signal pro 
cessing usually begins with PAM. To use PDM or PPM in such an application, we have 
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to ensure that full-scale modulation will not cause a pulse from one message signal to enter 
a time slot belonging to another message signal. This restriction results in a wasteful use 
of time space in telephone systems that are characterized by high peak factors, which is 
one reason for not using PDM or PPM in telephony. Also, despite the fact that PPM is 
more efficient than PDM, they both fall short of the ideal system for exchanging trans- 
mission bandwidth for improved noise performance. 

Digital pulse modulation systems transmit analog message signals as a sequence of 
coded pulses, which is made possible through the combined use of sampling and quanti- 
zation. Pulse-code modulation is an important form of digital pulse modulation that is 
endowed with some unique system advantages, which, in turn, have made it the standard 
method of modulation for the transmission of such analog signals as voice and video 
signals. The advantages of pulse-code modulation include robustness to noise and inter- 
ference, efficient regeneration of the coded pulses along the transmission path, and a uni- 
form format for different kinds of baseband signals. 

Indeed, it is because of this list of advantages unique to pulse-code modulation that 
it has become the method of choice for the construction of public switched telephone 
networks (PSTNs). In this context, the reader should carefully note that the telephone 
channel viewed from the PSTN to an Internet service provider, for example, is nonlinear 
due to the use of companding and, most importantly, it is entirely digital. This observation 
has a significant impact on the design of high-speed modems for communication between 
a computer user and server, as discussed in Chapter 6. 

Delta modulation and differential pulse-code modulation are two other useful forms 
of digital pulse modulation. The principal advantage of delta modulation is the simplicity 
of its circuitry. In contrast, differential pulse-code modulation employs increased circuit 
complexity to reduce channel bandwidth. The improvement is achieved by using the idea 
of prediction to remove redundant symbols from an incoming data stream. A further 
improvement in the operation of differential pulse-code modulation can be made through 
the use of adaptivity to account for statistical variations in the input data, By so doing, 
bandwidth requirement is reduced significantly without serious degradation in system 
performance. 

Unlike adaptive differential pulse-code modulation, the MPEG audio coding stan- 
dard achieves the compression of stereophonic audio signals in a transparent, perceptually 
lossless manner. This impressive performance is realized by exploiting certain psycho- 
acoustic properties of the auditory system. 

At this point in the discussion, it is informative to take a critical look at the different 
forms of pulse modulation that we have described in this chapter. In a strict sense, the 
term pulse modulation is a misnomer in that all of its different forms, be they analog or 
digital, are in fact source coding techniques. We say this for the simple reason that a 
message signal remains a baseband signal after undergoing all the changes involved in a 
pulse modulation process. The baseband nature of a pulse-modulated signal is exemplified 
by the fact that, irrespective of its exact description, it can be transmitted over a baseband 
channel of adequate bandwidth. Indeed, the material presented in the next chapter is 
devoted to the baseband transmission of data represented by a sequence of pulses. 

It is also important to recognize that pulse modulation techniques are lossy in the 
sense that some information is lost as a result of the signal representation that they perform. 
For example, in pulse-amplitude modulation, the customary practice is to use low-pass 
anti-alias filtering prior to sampling; in so doing, information is lost by virtue of the fact 
that high-frequency components considered to be unessential are removed by the filter. 
The lossy nature of pulse modulation is most vividly seen in pulse-code modulation that 
is characterized by the generation of quantization noise (i.e., distortion); the transmitted 
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sequence of encoded pulses does not have the infinite precision needed to represent cop. 
tinuous samples exactly. Nevertheless, the loss of information incurred by the use of 4 
pulse-modulation process is under the designer's control in that it can be made smal] 
enough for it to be nondiscernible by the end user. 


The material presented in this chapter on pulse modulation has been from a signal 


processing perspective. We will revisit pulse-code modulation in Chapter 9, which is de. 
voted to information-theoretic considerations of communication systems. In so doing we 
will develop deeper insight into its operation as a source coding technique. 


i NOTES AND REFERENCES 


1. 


10. 


11. 


The classic book on pulse modulation is Black (1953). A more detailed treatment of the 
subject is presented in the book by Rowe (1965). For the noise analysis of a PPM system, 
see the third edition of the book by Haykin (1994). 


. Pulse-code modulation was invented by Reeves in 1937. For a historical account of this 


invention, see the paper by Reeves (1975). The book by Jayant and Noll (1984) presents 
detailed treatment of pulse-code modulation, differential pulse-code modulation, delta 
modulation, and their variants. The book edited by Jayant (1976) provides a collection of 
early papers written on waveform quantization and coding. 


. For a detailed discussion of quantization noise in PCM systems, see the paper by Bennett 


(1948) and also the book by Rowe (1965, pp. 311-321). 


. The two necessary conditions of Equations (3.42) and (3.47) for optimality of a scalar 


quantizer were reported independently by Lloyd (1957) and Max (1960), hence the name 
“Lloyd-Max quantizer.” The derivation of these two optimality conditions presented in 
this chapter follows the book by Gersho and Gray (1992). 


. The p-law used for signal compression is described in Smith (1957).The p-law is used in 


the United States, Canada, and Japan. In Europe, the A-law is used for signal compression; 
this compression law is described in Cattermode (1969, pp. 133-140). For a discussion of 
the p-law and A-law, see also the paper by Kaneko (1970). 


. For a description of the original version of the T1-carrier system, see the paper by Fultz 


and Penick (1965). The description given in Example 3.2 is based on an updated version 
of this system; see Henning and Pan (1972). 


. The North America/Japan standards for digital multiplexers were originally adopted by 


AT&T. Another set of standards has been adopted by CCITT for the rest of the world. 
The CCITT digital hierarchy is similar to that described in Section 3.10, except for certain 
changes in the specifications of the number of channel inputs to the five digital multiplexers 
and their individual bit rates. For details of the CCITT digital hierarchy, see Couch (1997). 


. For the original papers on delta modulation, see Schouten, DeJager, and Greefkes (1952) 


and DeJager (1952). For a review paper on delta modulation, see the paper by Schindler 
(1970). 


. Delta-sigma modulation is described in the book by Jayant and Noll (1984, pp. 399—400); 


see also the paper by Inose, Yasuda, and Murakami (1962). 


Differential pulse-code modulation was invented by Cutler; the invention is described in 4 
patent issued in 1952. For a comparison of the noise performances of PCM and DPCM, 
see the paper by Jayant (1974); see also Rabiner and Schafer (1978, Chapter 5). 


For a discussion of adaptive differential pulse-code modulation, see Jayant and Noll (1984) 
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12. The adaptive delta modulation algorithm (ADM) of Equation (3.84) is the corrected ver- 
sion of an algorithm presented in Sklar (1988, p. 641). Sklar’s algorithm was adapted from 
an earlier paper by Song et al. (1971), where an optimum ADM system is derived; the 
highly nonlinear equations characterizing the optimum system are approximated in the 
latter paper by piecewise-linear equations for the purpose of implementation. 


13. The MPEG-1/audio coding standard is described in the papers by Brandenburg and Stoll 
(1994), Pan (1993), and the article by Peter Noll in the handbook on Digital Signal Pro- 
cessing edited by Madisetti and Williams (1998); the latter article also discusses the follow- 
up standards to MPEG-1. In particular, the MPEG-2 offers stereophonic audio coding at 
sampling rates lower than MPEG-1. 


14. The ear, the organ of hearing, responds to incoming acoustical waves. It has three main 
J 8 . 8 pon 8 
parts, with their functions as summarized here: 


> The outer ear aids in the collection of sounds. 

%» The middle ear provides an acoustic impedance match between the air and the cochlea fluids, 
thereby conveying the vibrations of the tympanic membrane (eardrum) due to the incoming 
sounds to the inner ear in an efficient manner. 

œ The inner ear converts the mechanical vibrations from the middle ear to an electrochemical 
or neural signal for transmission to the brain for processing. 


l PROBLEMS 


Sampling Process 


3.1 A narrowband signal has a bandwidth of 10 kHz centered on a carrier frequency of 
100 kHz. It is proposed to represent this signal in discrete-time form by sampling its in- 
phase and quadrature components individually. What is the minimum sampling rate that 
can be used for this representation? Justify your answer. How would you reconstruct the 
original narrowband signal from the sampled versions of its in-phase and quadrature 
components? 

3.2 In natural sampling, an analog signal g(t) is multiplied by a periodic train of rectangular 
pulses c(t). Given that the pulse repetition frequency of this periodic train is f, and the 
duration of each rectangular pulse is T (with f,T << 1), do the following: 

(a) Find the spectrum of the signal s(t) that results from the use of natural sampling; you 
may assume that time t = 0 corresponds to the midpoint of a rectangular pulse in 
c(t). 

(b) Show that the original signal z(t) may be recovered exactly from its naturally sampled 
version, provided that the conditions embodied in the sampling theorem are satisfied, 


3.3 Specify the Nyquist rate and the Nyquist interval for each of the following signals: 
(a) g(t) = sinc(200¢) 
(b) g(t) = sinc?(2002) 
(c) g(t) = sinc(200#) + sinc?(200t) 
3.4 (a) Plot the spectrum of a PAM wave produced by the modulating signal 
m(t) = Åm cos(27f,,t) 


assuming a modulation frequency f,, = 0.25 Hz, sampling period T, = 1 s, and pulse 
duration T = 0.45 s. 

(b) Using an ideal reconstruction filter, plot the spectrum of the filter output. Compare 
this result with the output that would be obtained if there were no aperture effect. 
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Pulse-Amplitude Modulation 


3.5 Figure P3.5 shows the idealized spectrum of a message signal m{ż). The signal is sampled 
at a rate equal to 1 kHz using flat-top pulses, with each pulse being of unit amplitude 
and duration 0.1 ms. Determine and sketch the spectrum of the resulting PAM signal, 


IMAI 
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FIGURE P3.5 


3.6 In this problem, we evaluate the equalization needed for the aperture effect in a PAM 
system. The operating frequency f = ,/2, which corresponds to the highest frequency 
component of the message signal for a sampling rate equal to the Nyquist rate. Plot 
1/sinc (0.5T/T,) versus T/T,, and hence find the equalization needed when T/T, = 0.1. 

3.7 Consider a PAM wave transmitted through a channel with white Gaussian noise and 
minimum bandwidth Br = 1/2T., where T, is the sampling period. The noise is of zero 
mean and power spectral density No/2. The PAM signal uses a standard pulse g(t) with 
its Fourier transform defined by 


1 
Gf) = 4 2B," |f| < Br 
0, |f| > Br 


By considering a full-load sinusoidal modulating wave, show that PAM and baseband- 
signal transmission have equal signal-to-noise ratios for the same average transmitted 
power. 

3.8 Twenty-four voice signals are sampled uniformly and then time-division multiplexed. The 
sampling operation uses flat-top samples with 1 us duration. The multiplexing operation 
includes provision for synchronization by adding an extra pulse of sufficient amplitude 
and also 1 ps duration. The highest frequency component of each voice signal is 3.4 kHz. 
(a) Assuming a sampling rate of 8 kHz, calculate the spacing between successive pulses 

of the multiplexed signal. 
(b) Repeat your calculation assuming the use of Nyquist rate sampling. 

3.9 Twelve different message signals, each with a bandwidth of 10 kHz, are to be multiplexed 
and transmitted. Determine the minimum bandwidth required for each method if the 
multiplexing/modulation method used is 
(a) FDM, SSB. 

(b) TDM, PAM. 

3.10 A PAM telemetry system involves the multiplexing of four input signals: s,(¢), i= 1, 2, 3, 
4. Two of the signals s,(£) and s2(ż) have bandwidths of 80 Hz each, whereas the remaining 
two signals s;(¢) and s4(t) have bandwidths of 1 kHz each, The signals s3(z) and s4(¢) ate 
each sampled at the rate of 2400 samples per second. This sampling rate is divided by 2" 
(i.e., an integer power of 2) to derive the sampling rate for s,(t) and s(t). 

(a) Find the maximum value of R. 

(b) -Using the value of R found in part (a), design a multiplexing system that first multi- 
plexes s,(f) and s(t) into a new sequence, ss(t), and then multiplexes s3(ż), s4(t), and 
ss(t}). 
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Line Codes 


3.11 


3.12 


3.13 


In this problem we derive the formulas used to compute the power spectra of Figure 3.16 
for the five line codes described in Section 3.7. In the case of each line code, the bit 
duration is T, and the pulse amplitude A is conditioned to normalize the average power 
of the line code to unity as indicated in Figure 3.16. Assume that the data stream is 
tandomly generated, and symbols 0 and 1 are equally likely. 

Derive the power spectral densities of these line codes as summarized here: 


(a) Unipolar nonreturn-to-zero signals: 


2 
sf) = sinc (1 +E ain} 





(b) Polar nonreturn-to-zero signals: 
S(f) = AT, sinc?(fT;) 


(c) Unipolar return-to-zero signals: 


S(f) = SS sine?) + Š > a(r - *)| 


(d) Bipolar return-to-zero signals: 








Sif) = = sine) sin*(afT;) 


(e) Manchester-encoded signals: 


S(f) = AT; sin (£2) sot 22) 
2 2 

Hence, confirm the spectral plots displayed in Figure 3.16. 
Suppose a random binary data stream (with equiprobable symbols) is differentially en- 
coded and then transmitted using one of the five line codes described in Problem 3.11. 
How is the power spectral density of the transmitted data affected by the use of differential 
encoding? Justify your answer. 
A randomly generated data stream consists of equiprobable binary symbols 0 and 1. It is 
encoded into a polar nonreturn-to-zero waveform with each binary symbol being defined 


as follows: 
at T, <t< T, 
st) = 4T, aa 


0, otherwise 





(a) Sketch the waveform so generated, assuming that the data stream is 00101110. 

(b) Derive an expression for the power spectral density of this signal, and sketch it. Hint: 
use Equation (1.52). 

(c) Compare the power spectral density of this random waveform with that defined in 
part (b) of Problem 3.11. 


Given the data stream 1110010100, sketch the transmitted sequence of pulses for each 
of the following line codes: 


(a) Unipolar nonreturn-to-zero 
(b) Polar nonreturn-to-zero 

(c) Unipolar return-to-zero 

(d) Bipolar return-to-zero 

(e) Manchester code 
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3.15 Suppose the binary data stream considered in Problem 3.14 is differentially encoded ang 
then transmitted using one of the five line codes considered therein. Sketch each of the 
transmitted data streams, assuming the use of symbol 1 for the reference bit. How is the 
result affected if symbol 0 is used for the reference bit? 


Pulse-Code Medulation 


3.16 A speech signal has a total duration of 10 s. It is sampled at the rate of 8 kHz and then 
encoded. The signal-to-(quantization) noise ratio is required to be 40 dB. Calculate the 
minimum storage capacity needed to accommodate this digitized speech signal. 

3.17 Consider a uniform quantizer characterized by the input-output relation illustrated jy 
Figure 3.104. Assume that a Gaussian-distributed random variable with zero mean and 
unit variance is applied to this quantizer input. 

(a) What is the probability that the amplitude of the input lies outside the range ~4 to 
+4? 

(b) Using the result of part (a), show that the output signal-to-noise ratio of the quantizer 
is given by 


(SNR)o = 6R — 7.2 dB 


where R is the number of bits per sample. Specifically, you may assume that the 
quantizer input extends from —4 to +4. Compare the result of part (b) with that 
obtained in Example 3.1. 

3.18 A PCM system uses a uniform quantizer followed by a 7-bit binary encoder. The bit rate 
of the system is equal to 50 X 10° b/s. 

(a) What is the maximum message bandwidth for which the system operates 
satisfactorily? 

(b) Determine the output signal-to-(quantization) noise ratio when a full-load sinusoidal 
modulating wave of frequency 1 MHz is applied to the input. 

3.19 Show that, with a nonuniform quantizer, the mean-square value of the quantization error 
is approximately equal to (1/12)3; A?p,, where A; is the ith step size and p; is the proba- 
bility that the input signal amplitude lies within the ith interval. Assume that the step size 
A, is small compared with the excursion of the input signal. 


3.20 (a) A sinusoidal signal, with an amplitude of 3.25 volts, is applied to a uniform quantizer 
of the midtread type whose output takes on the values 0, +1, +2, +3 volts. Sketch 
the waveform of the resulting quantizer output for one complete cycle of the input. 

(b) Repeat this evaluation for the case when the quantizer is of the midrise type whose 
output takes on the values =0.5, +1.5, +2.5, +3.5 volts. 


3.21 The signal 
m(t) = 6 sin(27t) volts 


is transmitted using a 4-bit binary PCM system. The quantizer is of the midrise type, with 
a step size of 1 volt. Sketch the resulting PCM wave for one complete cycle of the input. 
Assume a sampling rate of four samples per second, with samples taken at t = +1/8, 
+3/8, +5/8, . . . , seconds. 


3.22 Figure P3.22 shows a PCM signal in which the amplitude levels of +1 volt and -1 volt 
are used to represent binary symbols 1 and 0, respectively. The code word used consist 
of three bits. Find the sampled version of an analog signal from which this PCM signal 
is derived. 





FIGURE P3.22 


3.23 Consider a chain of (# — 1) regenerative repeaters, with a total of # sequential decisions 
made on a binary PCM wave, including the final decision made at the receiver. Assume 
that any binary symbol transmitted through the system has an independent probability 
pı of being inverted by any repeater. Let p,, represent the probability that a binary symbol 
is in error after transmission through the complete system. 


(a) Show that 
Pn = 3[1 — (1 — 2p,)7] 


(b) If p, is very small and # is not too large, what is the corresponding value of p„? 


3.24 Discuss the basic issues involved in the design of a regenerative repeater for pulse-code 
modulation. 


Delta Modulation 
3.25 Consider a test signal m(ż) defined by a hyperbolic tangent function: 


m(t) = A tanh(ft) 


where A and £ are constants. Determine the minimum step size A for delta modulation 
of this signal, which is required to avoid slope overload. 


3.26 Consider a sine wave of frequency f,, and amplitude Am, which is applied to a delta 
modulator of step size A. Show that slope-overload distortion will occur if 


A 
> panen 
Am > TaT, 


where T, is the sampling period. What is the maximum power that may be transmitted 
without slope-overload distortion? 

3.27 A linear delta modulator is designed to operate on speech signals limited to 3.4 kHz. The 
specifications of the modulator are as follows: 
> Sampling rate = 10fxyquisc, Where fryquist is the Nyquist rate of the speech signal. 
e Step size A = 100 mV. 


The modulator is tested with a 1-kHz sinusoidal signal. Determine the maximum ampli- 
tude of this test signal required to avoid slope overload. 

3.28 In this problem, we derive an empirical formula for the average signal-to-(quantization) 
noise ratio of a DM system with a sinusoidal signal of amplitude A and frequency f,, as 
the test signal. Assume that the power spectral density of the granular noise generated by 
the system is governed by the formula 


2 
SrA f) = a 


where f, is the sampling rate and A is the step size. (Note that this formula is basically 
the same as that for the power spectral density of quantization noise in a PCM system 
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with A/2 for PCM being replaced by A for DM.) The DM system is designed to handie 
analog message signals limited to bandwidth W. 
(a) Show that the average quantization noise power produced by the system is 
_ 4TA fW 
af 
where it is assumed that the step size A has been chosen in accordance with the 
formula used in Problem 3.27 so as to avoid slope overload. 
(b) Hence determine the signal-to-(quantization) noise ratio of the DM system for a 
sinusoidal input. 

3.29 Consider a DM system designed to accommodate analog message signals limited to band. 
width W = 5 kHz. A sinusoidal test signal of amplitude A = 1 volt and frequency 
fm = 1 kHz is applied to the system. The sampling rate of the system is 50 kHz. 

(a) Calculate the step size A required to minimize slope overload. 

(b) Calculate the signal-to-(quantization) noise ratio of the system for the specified 
sinusoidal test signal. 

For these calculations, use the formulas derived in Problems 3.27 and 3.28. 

3.30 Consider a low-pass signal with a bandwidth of 3 kHz. A linear delta modulation system, 
with step size A = 0.1V, is used to process this signal at a sampling rate ten times the 
Nyquist rate. 

(a) Evaluate the maximum amplitude of a test sinusoidal signal of frequency 1 kHz, 
which can be processed by the system without slope-overload distortion. 

(b) For the specifications given in part (a), evaluate the output signal-to-noise ratio under 
(i) prefiltered, and (ii) postfiltered conditions. 


Linear Prediction 


3.31 A one-step linear predictor operates on the sampled version of a sinusoidal signal. The 
sampling rate is equal to 10fy where fo is the frequency of the sinusoid. The predictor 
has a single coefficient denoted by w1. 

(a) Determine the optimum value of w, required to minimize the prediction error 
variance. 
(b) Determine the minimum value of the prediction error variance. 
3.32. A stationary process X(t) has the following values for its autocorrelation function: 


Rx(0) = 1 
Rx(1) = 0.8 
Rx(2) = 0.6 
Rx(3) = 0.4 


(a) Calculate the coefficients of an optimum linear predictor involving the use of three 
unit-delays. 
(b) Calculate the variance of the resulting prediction error. 
3.33 Repeat the calculations of Problem 3.32, but this time use a linear predictor with two 
unit-delays. Compare the performance of this second optimum linear predictor with that 
considered in Problem 3.32. 


Differential Pulse-Code Modulation 


3.34 A DPCM system uses a linear predictor with a single tap. The normalized autocorrelation 
function of the input signal for a lag of one sampling interval is 0.75. The predictor 8 
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designed to minimize the prediction error variance. Determine the processing gain attained 
by the use of this predictor. 

3.35 Calculate the improvement in processing gain of a DPCM system using the optimized 
three-tap linear predictor of Problem 3.32 over that of the optimized two-tap linear pre- 
dictor of Problem 3.33. For this calculation, use the autocorrelation function values of 
the input signal specified in Problem 3.32. 

3.36 In this problem, we compare the performance of a DPCM system with that of an ordinary 
PCM system using companding. 

For a sufficiently large number of representation levels, the signal-to-(quantization) 
noise ratio of PCM systems, in general, is defined by 


10 logio(SNR)o = a + 6n dB 


where 2” is the number of representation levels. For a companded PCM system using the 
p-law, the constant a is itself defined by 


a = 4.77 — 20 logio(log(1 + )) dB 


For a DPCM system, on the other hand, the constant æ lies in the range —3 < a < 15 
dBs. The formulas quoted herein apply to telephone-quality speech signals. 
Compare the performance of the DPCM system against that of the u-companded 
PCM system with u = 255 for each of the following scenarios: 
(a) The improvement in (SNR)o realized by DPCM over companded PCM for the same 
number of bits per sample. 
(b) The reduction in the number of bits per sample required by DPCM, compared to the 
companded PCM for the same (SNR)o. 
3.37 In the DPCM system depicted in Figure P3.37, show that in the absence of channel noise, 
the transmitting and receiving prediction filters operate on slightly different input signals. 
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FIGURE P3.37 














Computer Experiments 

3.38 A sinusoidal signal of frequency fọ = 104/27 Hz is sampled at the rate of 8 kHz and then 
applied to a sample-and-hold circuit to produce a flat-topped PAM signal s(t) with pulse 
duration T = 500 ps. 
(a) Compute the waveform of the PAM signal s(t). 
(b) Compute | S(f)|, denoting the magnitude spectrum of the PAM signal s(t). 


(c) Compute the envelope of |S(f)|. Hence confirm that the frequency at which this 
envelope goes through zero for the first time is equal to (1/T) = 20 kHz. 


3.39 In this problem, we use computer simulation to compare the performance of a companded 
PCM system using the u-law against that of the corresponding system using a uniform 
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3.40 


quantizer. The simulation is to be performed for a sinusoidal input signal of Varying 


„amplitude. 


(a) Using the p-law described in Table 3.4, plot the output signal-to-noise ratio as a 
function of the input signal-to-noise ratio, both ratios being expressed in decibels, 


(b) Compare the results of your computation in part (a) with a uniform quantizer having 
256 representation levels. 
In this experiment we study the linear adaptive prediction of a signal x[#] governed by 
the following recursion: ` 
x[n] = 0.8x[n — 1] — 0.1x[n — 2] + 0.1v[a] 


where v[n] is drawn from a discrete-time white noise process of zero mean and unit 
variance. (A process generated in this manner is referred to as an autoregressive process 
of order two.) Specifically, the adaptive prediction is performed using the normalized LMg 
algorithm defined by 


p 
sin] = $ wilnleln — k] 


eln] = Xir] — ŝia] 


win + 1] = wiln] + { —*+ | an — klein, k= 1,2,...,p 


> x[n- k] 


k=1 


where p is the prediction order and pris the normalized step-size parameter. The important 
point to note here is that yz is dimensionless and stability of the algorithm is assured by 
choosing it in accordance with the formula 


O<p<2 
The algorithm is initiated by setting 
w,[0] = 0 forallk 


The learning curve of the algorithm is defined as a plot of the mean-square error 
versus the number of iterations » for specified parameter values, which is obtained by 
averaging the plot of e[n] versus n over a large number of different realizations of the 
algorithm. 

(a) Plot the learning curves for the adaptive prediction of x[7] for a fixed prediction order 
p = 5 and three different values of step-size parameter: p = 0.0075, 0.05, and 0.5. 
(b) What observations can you make from the learning curves of part (a)? 






BASEBAND PULSE 
TRANSMISSION 


This chapter discusses the transmission of digital data over a baseband channel, with 
emphasis on the following topics: 


> The matched filter, which is the optimum system for detecting a known signal in additive 
white Gaussian noise. 


p Calculation of the bit error rate due to the presence of channel noise. 


> Intersymbol interference, which arises when the channel is dispersive as is commonly the 
case in practice. 


> Nyquist’s criterion for distortionless baseband data transmission. 


> Correlative-level coding or partial-response signaling for combatting the effects of 
intersymbol interference. 


> Digital subscriber lines. 
> Equalization of a dispersive baseband channel. 


> The eye pattern for displaying the combined effects of intersymbol interference and 
channel noise in data transmission. 


i 4.1 Introduction 


In Chapter 3 we described techniques for converting an analog information-bearing signal 
into digital form. There is another way in which digital data can arise in practice: The 
data may represent the output of a source of information that is inherently discrete in 
nature (e.g., a digital computer). In this chapter we study the transmission of digital data 
(of whatever origin) over a baseband channel.' Data transmission over a band-pass channel 
using modulation is covered in Chapter 6. 

Digital data have a broad spectrum with a significant low-frequency content. Base- 
band transmission of digital data therefore requires the use of a low-pass channel with a 
bandwidth large enough to accommodate the essential frequency content of the data 
stream. Typically, however, the channel is dispersive in that its frequency response deviates 
from that of an ideal low-pass filter. The result of data transmission over such a channel 
is that each received pulse is affected somewhat by adjacent pulses, thereby giving rise to 
a common form of interference called intersymbol interference (ISI). Intersymbol interfer- 
ence is a major source of bit errors in the reconstructed data stream at the receiver output. 
To correct for it, control has to be exercised over the pulse shape in the overall system. 
Thus much of the material covered in this chapter is devoted to pulse shaping in one form 
or another. 
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Another source of bit errors in a baseband data transmission system is the ubiquitoy, 
channel noise. Naturally, noise and ISI arise in the system simultaneously. However, to 
understand how they affect the performance of the system, we first consider them sepa. 
rately; later on in the chapter, we study their combined effects. 

We thus begin the chapter by describing a fundamental result in communication 
theory, which deals with the detection of a pulse signal of known waveform that is im. 
mersed in additive white noise. The device for the optimum detection of such a pulse 
involves the use of a linear-time-invariant filter known as a matched filter, which is so 
called because its impulse response is matched to the pulse signal. 


i 4.2 Matched Filter 


A basic problem that often arises in the study of communication systems is that of detecting 
a pulse transmitted over a channel that is corrupted by channel noise (i.e., additive noise 
at the front end of the receiver). For the purpose of the discussion presented in this section, 
we assume that the major source of system limitation is the channel noise. 

Consider then the receiver model shown in Figure 4.1, involving a linear time-invar- 
iant filter of impulse response k(t). The filter input x(t) consists of a pulse signal g(t) 
corrupted by additive channel noise w(t), as shown by 


x(t) = g(t) + w(t) O=t=T (4.1) 


where T is an arbitrary observation interval. The pulse signal g(t) may represent a binary 
symbol 1 or 0 in a digital communication system. The w(t) is the sample function of a 
white noise process of zero mean and power spectral density No/2. It is assumed that the 
receiver has knowledge of the waveform of the pulse signal g(t). The source of uncertainty 
lies in the noise w(t). The function of the receiver is to detect the pulse signal g(t) in an 
optimum manner, given the received signal x(t). To satisfy this requirement, we have to 
optimize the design of the filter so as to minimize the effects of noise at the filter output 
in some statistical sense, and thereby enhance the detection of the pulse signal g(t). 
Since the filter is linear, the resulting output y(t) may be expressed as 


y(t) = golt) + n(t) (4.2) 


where g,(¢) and n(t) are produced by the signal and noise components of the input x(t), 
respectively. A simple way of describing the requirement that the output signal component 
golt) be considerably greater than the output noise component x(t) is to have the filter 
make the instantaneous power in the output signal g,(t), measured at time t = T, as large 
as possible compared with the average power of the output noise 7(t). This is equivalent 
to maximizing the peak pulse signal-to-noise ratio, defined as 


z |g(T) | $ (4.3) 


7° Epee] 
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FIGURE 4.1 Linear receiver. 
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where |g,(T) |? is the instantaneous power in the output signal, E is the statistical expec- 
tation operator, and E[n°(z)] is a measure of the average output noise power. The require- 
ment is to specify the impulse response h(t) of the filter such that the output signal-to- 
noise ratio in Equation (4.3) is maximized. 

Let G(f) denote the Fourier transform of the known signal g(t), and H(f) denote 
the frequency response of the filter. Then the Fourier transform of the output signal g,(2) 
is equal to H(f)G(f), and g.(t) is itself given by the inverse Fourier transform 


g(t) = T H(f)G(f) exp( j2aft) df (4.4) 


Hence, when the filter output is sampled at time t = T, we have (in the absence of channel 
noise) 
2 


lg(T)|? = iz H(f)G(f) exp(j2afT) df (4.5) 





Consider next the effect on the filter output due to the noise w(t) acting alone. The 
power spectral density $x (f) of the output noise z(t) is equal to the power spectral density 
of the input noise w(t) times the squared magnitude response |H(f)|? (see Section 1.7). 
Since 1(z) is white with constant power spectral density No/2, it follows that 


suf) = ~ | HUA? (4.6 


The average power of the output noise z(t) is therefore 


EI = | Ssif) df 
(4.7) 


N, f? 

=% f paf 
Thus substituting Equations (4.5) and (4.7) into (4.3), we may rewrite the expression 

for the peak pulse signal-to-noise ratio as 

2 





f HEGIP expiaapr) df 











(4.8) 


Xo f (HUN)? df 
Our problem is to find, for a given G(f), the particular form of the frequency response 
H(f) of the filter that makes ņ a maximum. To find the solution to this optimization 
problem, we apply a mathematical result known as Schwarz’s inequality to the numerator 
of Equation (4.8). 

A derivation of Schwarz’s inequality is given in Chapter 5. For now it suffices to say 
that if we have two complex functions ¢,(x) and (x) in the real variable x, satisfying 
the conditions 


E | @ilx)|? dx < œ 


and 


f loal dx < = 
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then we may write 
2 


k $ ı(x)ġ2(x) dx) = E | bax) |? dx ‘ie | b2(x) |? dx (4.3) 








The equality in (4.9) holds if, and only if, we have 
g(x) = ko Ze) (4.19 


where & is an arbitrary constant, and the asterisk denotes complex conjugation, 

Returning to the problem at hand, we readily see that by invoking Schwarz’s jp. 
equality (4.9), and setting ¢ ,(x) = H(f) and 2(x) = G(f) exp(jafT), the numerator in 
Equation (4.8) may be rewritten as ` 


æ 2 0 æ 
[anew etzan at) = f neat | IDRA a 








Using this relation in Equation (4.8), we may redefine the peak pulse signal-to-noise ratio 
as 


3 re 
1SN x IG(f)|? df (4.12) 


The right-hand side of this relation does not depend on the frequency response H(f) of 
the filter but only on the signal energy and the noise power spectral density. Consequently, 
the peak pulse signal-to-noise ratio 7 will be a maximum when H(f) is chosen so that the 
equality holds; that is, 


nw = | (GUI? af iy 


Correspondingly, H(f) assumes its optimum value denoted by H,,(f). To find this opti- 
mum value we use Equation (4.10), which, for the situation at hand, yields 


Hoplf) = RG*(f) exp(—j2afT) (4.14) 


where G*(f) is the complex conjugate of the Fourier transform of the input signal g(t), 
and k is a scaling factor of appropriate dimensions. This relation states that, except for 
the factor k exp(—j2afT), the frequency response of the optimum filter is the same as the 
complex conjugate of the Fourier transform of the input signal. 

Equation (4.14) specifies the optimum filter in the frequency domain. To characterize 
it in the time domain, we take the inverse Fourier transform of H,,,(f) in Equation (4.14) 
to obtain the impulse response of the optimum filter as 


bolt) = k E G*(f) exp[—j2af(T - t)] df (4,15) 


Since for a real signal g(t) we have G*(f) = G(—f), we may rewrite Equation (4.15) a8 


boplt) = k ix G(—f) exp[—j2mf(T — #)] df 


=e L G(f) exp [j2af(T — t)] df (4.16) 


= kg(T — t) 


Equation (4.16) shows that the impulse response of the optimum filter, except for the 
scaling factor k, is a time-reversed and delayed version of the input signal g(t); that $, 
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is “matched” to the input signal. A linear time-invariant filter defined in this way is called 
a matched filter. Note that in deriving the matched filter the only assumption we have 
made about the input noise w(t) is that it is stationary and white with zero mean and 
power spectral density No/2. In other words, no assumption was made on the statistics of 
the channel noise w(t). 


PROPERTIES OF MATCHED FILTERS 


We note that a filter, which is matched to a pulse signal g(t) of duration T, is characterized 
by an impulse response that is a time-reversed and delayed version of the input g(t), as 
shown by 
Doplt) = kg(T — t) 

In other words, the impulse response /,,,(t) is uniquely defined, except for the delay T and 
-the scaling factor k, by the waveform of the pulse signal g(t) to which the filter is matched. 
In the frequency domain, the matched filter is characterized by a frequency response that 
is, except for a delay factor, the complex conjugate of the Fourier transform of the input 
g(t), as shown by 


Hol f) = kG*(f) exp(—j2afT) 
The most important result in the calculation of the performance of signal processing sys- 
tems using matched filters is perhaps the following: 


The peak pulse signal-to-noise ratio of a matched filter depends only on the ratio of the 
signal energy to the power spectral density of the white noise at the filter input. 


To demonstrate this property, consider a filter matched to a known signal g(t). The Fourier 
transform of the resulting matched filter output go(t) is 


Golf) = Hop f)G(A) 
= kG*(f)G(f) exp(—j2afT) (4.17) 
= k|G(f)|* exp(—j2afT) 


Using Equation (4.17) in the formula for the inverse Fourier transform, we find that the 
matched filter output at time t = T is 


elt) = | Golf) expli2nfT) df 
=k f lair af 


According to Rayleigh’s energy theorem, the integral of the squared magnitude spectrum 
of a pulse signal with respect to frequency is equal to the signal energy E: 


e={ ena- f japas 
Hence 
8(T) = kE (4.18) 
Substituting Equation (4.14) into (4.7), we find that the average output noise power is 


Er = S apas 


= K NE/2 





(4.19) 
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where again we have made use of Rayleigh’s energy theorem. Therefore, the peak Pulse 
signal-to-noise ratio has the maximum value 


KEP _2E 
(NoE/2) No (4.20) 


Nmax 


From Equation (4.20) we see that dependence on the waveform of the input g(t) has been 
completely removed by the matched filter. Accordingly, in evaluating the ability of 4 
matched-filter receiver to combat additive white noise, we find that all signals that have 
the same energy are equally effective. Note that the signal energy E is in joules and the 
noise spectral density No/2 is in watts per Hertz, so that the ratio 2E/Np is dimensionless. 
however, the two quantities have different physical meaning. We refer to E/N, as the signal 
energy-to-noise spectral density ratio. 


> EXAMPLE 4.1 Matched Filter for Rectangular Pulse 


Consider a signal g(t) in the form of a rectangular pulse of amplitude A and duration T, ay 
shown in Figure 4.2a. In this example, the impulse response b(t) of the matched filter has 
exactly the same waveform as the signal itself. The output signal g(t) of the matched filter 
produced in response to the input signal g(ż) has a triangular waveform, as shown in Figure 
4.26. 

The maximum value of the output signal g,(t) is equal to kA?T, which is the energy of 
the input signal g(t) scaled by the factor k; this maximum value occurs at t = T, as indicated 
in Figure 4.26. 
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FIGURE 4.2 (a) Rectangular pulse. (b) Matched filter output. (c) Integrator output. 
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FIGURE 4.3 Integrate-and-dump circuit. 


For the special case of a rectangular pulse, the matched filter may be implemented using 
a circuit known as the integrate-and-dump circuit, a block diagram of which is shown in 
Figure 4.3. The integrator computes the area under the rectangular pulse, and the resulting 
output is then sampled at time t = T, where T is the duration of the pulse. Immediately after 
t = T, the integrator is restored to its initial condition; hence the name of the circuit. Figure 
4.2c shows the output waveform of the integrate-and-dump circuit for the rectangular pulse 
of Figure 4.24. We see that for 0 = t = T, the output of this circuit has the same waveform 
as that appearing at the output of the matched filter; the difference in the notations used to 
describe their peak values is of no practical significance. 4 


| 4.3 Error Rate Due to Noise 


In Section 3.8 we presented a qualitative discussion of the effect of channel noise on the 
performance of a binary PCM system. Now that we are equipped with the matched filter 
as the optimum detector of a known pulse in additive white noise, we are ready to derive 
a formula for the error rate in such a system due to noise. 

To proceed with the analysis, consider a binary PCM system based on polar non- 
return-to-zero (NRZ) signaling. In this form of signaling, symbols 1 and 0 are represented 
by positive and negative rectangular pulses of equal amplitude and equal duration. The 
channel noise is modeled as additive white Gaussian noise w(t) of zero mean and power 
spectral density No/2; the Gaussian assumption is needed for later calculations. In the 
signaling interval 0 = t = T;, the received signal is thus written as follows: 


+A + bo 
ae { A + w(t), symbol 1 was sent (4.21) 


—A + w(t), symbol 0 was sent 


where T; is the bit duration, and A is the transrnitted pulse amplitude. It is assumed that 
the receiver has acquired knowledge of the starting and ending times of each transmitted 
pulse; in other words, the receiver has prior knowledge of the pulse shape, but not its 
polarity. Given the noisy signal x(t), the receiver is required to make a decision in each 
signaling interval as to whether the transmitted symbol is a 1 or a 0. 

The structure of the receiver used to perform this decision-making process is shown 
in Figure 4.4. It consists of a matched filter followed by a sampler, and then finally a 
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FIGURE 4.4 Receiver for baseband transmission of binary-encoded PCM wave using polar NRZ 
signaling. 
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decision device. The filter is matched to a rectangular pulse of amplitude A and duration 
Tp, exploiting the bit-timing information available to the receiver. The resulting matche d 
filter output is sampled at the end of each signaling interval. The presence of channel Noise 
w(t) adds randomness to the matched filter output. 

Let y denote the sample value obtained at the end of a signaling interval. The sample 
value y is compared to a preset threshold A in the decision device. If the threshold y 
exceeded, the receiver makes a decision in favor of symbol 1; if not, a decision is made jn 
favor of symbol 0. We adopt the convention that when the sample value y is exactly equal 
to the threshold A, the receiver just makes a guess as to which symbol was transmitted. 
such a decision is the same as that obtained by flipping a fair coin, the outcome of which 
will not alter the average probability of error. 

There are two possible kinds of error to be considered: 


1. Symbol 1 is chosen when a 0 was actually transmitted; we refer to this error as an 
error of the first kind. 

2. Symbol 0 is chosen when a 1 was actually transmitted; we refer to this error as an 
error of the second kind. 


To determine the average probability of error, we consider these two situations separately, 
Suppose that symbol 0 was sent. Then, according to Equation (4.21), the received 

signal is 
xlt) = —-Atw(t) O=t=T, (4.22) 


Correspondingly, the matched filter output, sampled at time t = Ta, is given by (in light 
of Example 4.1 with RAT; set equal to unity for convenience of presentation) 


r x(t) dt 


0 


y 
(4.23) 


1 S 
-Å + Th w(t) dt 


which represents the sample value of a random variable Y. By virtue of the fact that the 
noise w(t) is white and Gaussian, we may characterize the random variable Y as follows: 
® The random variable Y is Gaussian distributed with a mean of —A. 
> The variance of the random variable Y is 


oy = EĻ(Y + A)’ 


= T al [ Í w(t)w(u) dt au 
Ty 


j (4.24) 


E[w(t)w(u)] dt du 


“7 i 
1 (Te (% 

= Flo h Ry(t, u) dt du 
b 


where Ryt, u) is the autocorrelation function of the white noise w(t). Since w(t) is white 
with a power spectral density No/2, we have 


Ry(t, u) = x 5(t — u) (4.25) 
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where 4(t — u) is a time-shifted delta function. Hence, substituting Equation (4.25) into 


(4.24) yields 
Ty (Tp 
oF = = 1 ri {% No 5 5(t — u) dt du 
(4.26) 


= 
where we have used the sifting property of the delta function and the fact that its area is 
unity. The conditional probability density function of the random variable Y, given that 
symbol 0 was sent, is therefore 
+ AP 
or ar) (4.27) 


frly|0) = +e (- 
ry VaNolT, PN NolTs 


„This function is plotted in Figure 4.5(a). Let pio denote the conditional probability of error, 

given that symbol 0 was sent. This probability is defined by the shaded area under the 
curve of fy(y|0) from the threshold A to infinity, which corresponds to the range of values 
assumed by y for a decision in favor of symbol 1. In the absence of noise, the matched 
filter output y sampled at time t = T; is equal to —A. When noise is present, y occasionally 
assumes a value greater than A, in which case an error is made. The probability of this 
error, conditional on sending symbol 0, is defined by 


Pio = P(y > A|symbol 0 was sent) 


= | fola (4.28) 


_(y + AY 
ale 20( NolTs *) ay 


At this point in the discussion we digress briefly and introduce the definition of the 
so-called complementary error function? 


erfc(u) = = im exp(—27) dz (4.29) 


which is closely related to the Gaussian distribution. For large positive values of u, we 
have the following upper bound on the complementary error function: 


ig 
erfc(u) < eet (4.30) 


fyli 
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FIGURE 4.5 Noise analysis of PCM system. (a) Probability density function of random variable Y 
at matched filter output when 0 is transmitted. (b) Probability density function of Y when 1 is 
transmitted. 
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To reformulate the conditional probability of error pio in terms of the complemen. 
tary error function, we first define a new variable 
ytA 


VN,/T, 


Accordingly, we may rewrite Equation (4.28) in the compact form 


1 = 2 
Pio = <a fana exp(—2°) dz 


ad e e(a ta ) man 
2 SVNT, 


Assume next that symbol 1 was transmitted. This time the Gaussian random variable 
Y represented by the sample value y of the matched filter output has a mean +A and 
variance N,/2T,. Note that, compared to the situation when symbol 0 was sent, the mean 
of the random variable Y has changed, but its variance is exactly the same as before. The 
conditional probability density function of Y, given that symbol 1 was sent, is therefore 





a 1 (y - A? 
fly 1) =o VaN,/T; eo No/T, ) (4.32) 


which is plotted in Figure 4.5b. Let po, denote the conditional probability of error, given 
that symbol 1 was sent. This probability is defined by the shaded area under the curve of 
fv(y|1) extending from —% to the threshold A, which corresponds to the range of values 
assumed by y for a decision in favor of symbol 0. In the absence of noise, the matched 
filter output y sampled at time t = T; is equal to +A. When noise is present, y occasionally 
assumes a value less than A, and an error is then made. The probability of this etror, 
conditional on sending symbol 1, is defined by 


Poi = P(y < A|symbol 1 was sent) 


A 
= K fr(y|1) dy (4.33) 
1 F ( = a) 
= —— exp| -=> | d 
V nNolTp -Tae No/T, 
To express fo, in terms of the complementary error function, this time we define a new 
variable 





z= AT 
V No/T, 
Accordingly, we may reformulate Equation (4.33) in the compact form 


1 f° 


= Vr (A-AVWVROT Sea) de 


Poi 
(4.34) 


ad ere( 4) 
2 WNIT: 
Having determined the conditional probabilities of error, pio and por, our next task 
is to derive the formula for the average probability of symbol error, denoted by P,. Here 


we note that these two possible kinds of error are mutually exclusive events in that if the 
receiver, at a particular sampling instant, chooses symbol 1, then symbol 0 is exclu 
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from appearing, and vice versa. Let pọ and p; denote the a priori probabilities of trans- 
mitting symbols 0 and 1, respectively. Hence, the average probability of symbol error P, 
in the receiver is given by 


P, = popio + pipor 





Po E) pa E) (4.35) 
= = erfc| ——— |} + © erfc 
2 VN/T; 2 V No/T, 


From Equation (4.35) we see that P, is in fact a function of the threshold à, which 
immediately suggests the need for formulating an optimum threshold that minimizes Pa 
For this optimization we use Leibniz’s rule. 

Consider the integral 

b(u) 


be f(z, u) dz 


Leibniz’s rule states that the derivative of this integral with respect to n is 


d Pe 7 db(u) dialu) "afle, u) 
a Lug F056) de = fibt u) FE — fiatu u) AE y f fe g 


For the problem at hand, we note from the definition of the complementary error function 
in Equation (4.29) that 


f(z, u) = 2 exp(—z*) 
a(u) =u 
b(u) = œ 
The application of Leibniz’s rule to the complementary error function thus yields 
5 erfc(u) = = exp(—?) (4.36) 
m 


Hence, differentiating Equation (4.35) with respect to A by making use of the formula in 
Equation (4.36), then setting the result equal to zero and simplifying terms, we obtain the 
optimum threshold as 





-No Po 
den © Gace log(2?) (4.37) 


For the special case when symbols 1 and 0 are equiprobable, we have 
EAEAN. 
pı = Po 2 
in which case Equation (4.37) reduces to 
À opt = 0 
This result is intuitively satisfying as it states that, for the transmission of equiprobable 
binary symbols, we should choose the threshold at the midpoint between the pulse heights 


~A and +A representing the two symbols 0 and 1. Note that for this special case we also 
have 


Poi = Paio 
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A channel for which the conditional probabilities of error poi and fio are equal is said to 
be binary symmetric. Correspondingly, the average probability of symbol error in Equation 
(4.35) reduces to 


P, = 1 a(r) , 
e 2 4 INIT, (4.38) 
Now the transmitted signal energy per bit is defined by 
E, = A?°T, (4.39) 


Accordingly, we may finally formulate the average probability of symbol error for the 


receiver in Figure 4.4 as 
1 JE, 
P: = 5 erfe( 2) (4.40) 


which shows that the average probability of symbol error in a binary symmetric channel 
depends solely on E Nao, the ratio of the transmitted signal energy per bit to the noise 
spectral density. 

Using the upper bound of Equation (4.30) on the complementary error function, we 
may correspondingly bound the average probability of symbol error for the PCM receiver 
as 


exp(—E,/No) 
2V nE No 


The PCM receiver of Figure 4.4 therefore exhibits an exponential improvement in the 
average probability of symbol error with increase in E,/No. 

This important result is further illustrated in Figure 4.6 where the average probability 
of symbol error P, is plotted versus the dimensionless ratio E,/No. In particular, we see 
that P, decreases very rapidly as the ratio E,/No is increased, so that eventually a very 
“small increase” in transmitted signal energy will make the reception of binary pulses 
almost error free, as discussed previously in Section 3.8. Note, however, that in practical 
terms the increase in signal energy has to be viewed in the context of the bias; for example, 


P. < .(4.41) 
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FIGURE 4.6 Probability of error in a PCM receiver. 
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a 3-dB increase in E,/Npo is much easier to implement when E; has a small value than when 
its value is orders of magnitude larger. 


i 4,4 Intersymbol Interference 


Input 

binary 

data 
ih} 


The next source of bit errors in a baseband-pulse transmission system that we wish to 
study is intersymbol interference (ISI), which arises when the communication channel is 
dispersive. First of all, however, we need to address a key question: Given a pulse shape 
of interest, how do we use it to transmit data in M-ary form? The answer lies in the use 
of discrete pulse modulation, in which the amplitude, duration, or position of the trans- 
mitted pulses is varied in a discrete manner in accordance with the given data stream. 
However, for the baseband transmission of digital data, the use of discrete pulse-amplitude 
modulation (PAM) is one of the most efficient schemes in terms of power and bandwidth 
utilization. Accordingly, we confine our attention to discrete PAM systems. We begin the 
study by first considering the case of binary data; later in the chapter, we consider the 
more general case of M-ary data. 

Consider then a baseband binary PAM system, a generic form of which is shown in 
Figure 4.7, The incoming binary sequence {b,} consists of symbols 1 and 0, each of du- 
ration T. The pulse-amplitude modulator modifies this binary sequence into a new se- 
quence of short pulses (approximating a unit impulse), whose amplitude a, is represented 
in the polar form 


ars i if symbol b, is 1 (4.42) 


—1 if symbol 8, is 0 


The sequence of short pulses so produced is applied to a transmit filter of impulse response 
g(t), producing the transmitted signal 


s(t) = = ag(t — kT,) (4.43) 


The signal s(t) is modified as a result of transmission through the channel of impulse 
response h(t). In addition, the channel adds random noise to the signal at the receiver 
input. The noisy signal x(t) is then passed through a receive filter of impulse response c(t). 
The resulting filter output y(t) is sampled synchronously with the transmitter, with the 
sampling instants being determined by a clock or timing signal that is usually extracted 
from the receive filter output. Finally, the sequence of samples thus obtained is used to 
reconstruct the original data sequence by means of a decision device. Specifically, the 
amplitude of each sample is compared to a threshold A. If the threshold A is exceeded, a 
decision is made in favor of symbol 1. If the threshold A is not exceeded, a decision is made 
in favor of symbol 0. If the sample amplitude equals the threshold exactly, the flip of a 
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FIGURE 4.7 Baseband binary data transmission system. 
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fair coin will determine which symbol was transmitted (i.e., the receiver simply makes : 
random guess). 
The receive filter output is written as 


y(t) = a È apl — RT») + alt) (4.44) 


where pw is a scaling factor, and the pulse p(t) is to be defined. To be precise, an arbitrary 
time delay to should be included in the argument of the pulse p(t — kT) in Equation (4,44) 
to represent the effect of transmission delay through the system. To simplify the exposition 
we have put this delay equal to zero in Equation (4.44) without loss of generality. 

The scaled pulse yp(t) is obtained by a double convolution involving the impuls 
response g(t) of the transmit filter, the impulse response h(t) of the channel, and the impulse 
response c(t) of the receive filter, as shown by 


upit) = g(t) X b(t) X clt) (4.45) 
where the star denotes convolution. We assume that the pulse p(t) is normalized by setting 
p(0) = 1 (4.46) 


which justifies the use of yas a scaling factor to account for amplitude changes incurred 
in the course of signal transmission through the system. 

Since convolution in the time domain is transformed into multiplication in the fre. 
quency domain, we may use the Fourier transform to change Equation (4.45) into the 
equivalent form : 


PIF) = GAHE) (4.47) 


where P(f), G(f), H(f), and C(f) are the Fourier transforms of p(t), g(t), b(t), and c(t), 
respectively. 

Finally, the term z(t) in Equation (4.44) is the noise produced at the output of the 
receive filter due to the channel noise w(t). It is customary to model w(t) as a white Gaus- 
sian noise of zero mean. 

The receive filter output y(t) is sampled at time t; = iT, (with i taking on integer 
values), yielding [in light of Equation (4.46)] 


yt) = wD apli- KTA + mle) 


(4.48) 


Ha; + p > ayp[(i— k)To] + n(ti) 


k+i 


In Equation (4.48), the first term pa; represents the contribution of the ith transmitted bit 
The second term represents the residual effect of all other transmitted bits on the decoding 
of the ith bit; this residual effect due to the occurrence of pulses before and after the 
sampling instant t; is called intersymbol interference (ISI). The last term m(t;) represents the 
noise sample at time ¢;. 

In the absence of both ISI and noise, we observe from Equation (4.48) that 


y(t) = pa; 
which shows that, under these ideal conditions, the ith transmitted bit is decoded correctly. 
The unavoidable presence of ISI and noise in the system, however, introduces errors 1 the 
decision device at the receiver output. Therefore, in the design of the transmit and rece? 


filters, the objective is to minimize the effects of noise and ISI and thereby deliver the digi 
data to their destination with the smallest error rate possible. 
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When the signal-to-noise ratio is high, as is the case in a telephone system, for ex- 
ample, the operation of the system is largely limited by ISI rather than noise; in other 
words, we may ignore x(t,). In the next couple of sections, we assume that this condition 
holds so that we may focus our attention on ISI and the techniques for its control. In 
particular, the issue we wish to consider is to determine the pulse waveform p(t) for which 
the ISI is completely eliminated. 


4.5 Nyquist’s Criterion for Distortionless 
| Baseband Binary Transmission 


Typically, the frequency response of the channel and the transmitted pulse shape are spec- 
ified, and the problem is to determine the frequency responses of the transmit and receive 
filters so as to reconstruct the original binary data sequence {b,}. The receiver does this by 
extracting and then decoding the corresponding sequence of coefficients, {a,}, from the 
output y(t). The extraction involves sampling the output y(t) at time t = iT,. The decoding 
requires that the weighted pulse contribution a,p(iT;, — kT,) for k = i be free from ISI 
due to the overlapping tails of all other weighted pulse contributions represented by k # i. 
This, in turn, requires that we control the overall pulse p(t), as shown by 


1, i=k 


0, itk em) 


p(iT, — RT») = l 
where p(0) = 1, by normalization. If p(t) satisfies the conditions of Equation (4.49), the 
receiver output y(t;) given in Equation (4.48) simplifies to (ignoring the noise term) 


y(t) = pa; — for alli 


which implies zero intersymbol interference. Hence, the two conditions of Equation (4.49) 
ensure perfect reception in the absence of noise. 

From a design point of view, it is informative to transform the conditions of Equation 
(4.49) into the frequency domain. Consider then the sequence of samples {p(#T;)}, where 
n = 0, +1, +2, +» +. From the discussion presented in Chapter 3 on the sampling process, 
we recall that sampling in the time domain produces periodicity in the frequency domain. 
In particular, we may write 


Pf) =R, > Pif- nR) (4.50) 


where R, = 1/T, is the bit rate in bits per second (b/s); Ps(f) is the Fourier transform of 
an infinite periodic sequence of delta functions of period T}, whose individual areas are 
weighted by the respective sample values of p(t). That is, P;(f) is given by 


PAf) = |S. [ew Ts) ate — mT4)] exp(—j2nfe) de (4.51) 


ma 


Let the integer » = i — k. Then, i = k corresponds to m = 0, and likewise i # k corresponds 
to m # 0. Accordingly, imposing the conditions of Equation (4.49) on the sample values 
of p(t) in the integral of Equation (4.51), we get 


Pf) = im (0) S(t) exp(—j2aft) dt 
= p(0) 


(4.52) 
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where we have made use of the sifting property of the delta function. Since from Equation 
(4.46) we have p(0) = 1, it follows from Equations (4.50) and (4.52) that the condition 
for zero intersymbol interference is satisfied if 


È Pif- #Rs) = Ts (4.53) 

We may now state the Nyquist criterion’ for distortionless baseband transmission in 

the absence of noise: The frequency function P(f) eliminates intersymbol interference fo 

samples taken at intervals T, provided that it satisfies Equation (4.53). Note that Pif) 

refers to the overall system, incorporating the transmit filter, the channel, and the receiyg 
filter in accordance with Equation (4.47). 


IDEAL NYQUIST CHANNEL 


The simplest way of satisfying Equation (4.53) is to specify the frequency function P(f) to 
be in the form of a rectangular function, as shown by 


1 


Pif) = IW’ -W<f<W 
0, iris Y (4.54) 
1 f 
= wwe) 


where rect(f) stands for a rectangular function of unit amplitude and unit support centered 
on f = 0, and the overall system bandwidth W is defined by 


-Re 1 
wes 2T, 


(4.55) 
According to the solution described by Equations (4.54) and (4.55), no frequencies of 
absolute value exceeding half the bit rate are needed. Hence, from Fourier-transform pair 
2 of Table A6.3 we find that a signal waveform that produces zero intersymbol interference 
is defined by the sinc function: 


= sin(27Wt) 
pl) 2nWt (4.56) 
= sinc(2 Wt) 


The special value of the bit rate R, = 2W is called the Nyquist rate, and W is itself 
called the Nyquist bandwidth. Correspondingly, the ideal baseband pulse transmission, 
system described by Equation (4.54) in the frequency domain or, equivalently, Equation 
(4.56) in the time domain, is called the ideal Nyquist channel. 

Figures 4.8a and 4.8b show plots of P(f) and p(t), respectively. In Figure 4.8a, the 
normalized form of the frequency function P(f) is plotted for positive and negative fre- 
quencies. In Figure 4.8b, we have also included the signaling intervals and the correspond- 
ing centered sampling instants. The function p(t) can be regarded as the impulse respons? 
of an ideal low-pass filter with passband magnitude response 1/2 W and bandwidth W. 
The function p(t) has its peak value at the origin and goes through zero at integer multiples 
of the bit duration Ts. It is apparent that if the received waveform y(t} is sampled at the 
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FIGURE 4.8 (a) Ideal magnitude response. (b) Ideal basic pulse shape. 


instants of time t = 0, +T,, +2T,,++-, then the pulses defined by pp(t — iT,) with 
arbitrary amplitude p and index i = 0, +1, +2, -++ , will not interfere with each other. 
This condition is illustrated in Figure 4.9 for the binary sequence 1011010. 

Although the use of the ideal Nyquist channel does indeed achieve economy in band- 
width in that it solves the problem of zero intersymbol interference with the minimum 


Binary sequence 1 0 1 1 0 
T ——=+ 





Amplitude 














FIGURE 4.9 A series of sinc pulses corresponding to the sequence 1011010. 
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bandwidth possible, there are two practical difficulties that make it an undesirable Objeg 
tive for system design: 


1. It requires that the magnitude characteristic of P(f) be flat from — W to W, and zerg 
elsewhere. This is physically unrealizable because of the abrupt transitions at th 
band edges + W. 

2. The function p(t) decreases as 1/|t| for large |t|, resulting in a slow rate of decay, 
This is also caused by the discontinuity of P(f) at +W. Accordingly, there is Pras. 
tically no margin of error in sampling times in the receiver. 


To evaluate the effect of this timing error, consider the sample of y(t) at t = Ar 
where At is the timing error. To simplify the exposition, we may put the correct sampling 
time t; equal to zero. In the absence of noise, we thus have (from Equation (4.48)) 


y(At) =p z a,p(At — kT;) 








5 a, SWA — kT,)] (4.57) 
P os InWAt — kT) 
Since 2WT, = 1, by definition, we may rewrite Equation (4.57) as 
Z . u sin(27W At) (-1)*a, 
y(At) = pao sinc(2W At) + = z OWA k) aH (4.58) 


k+0 


The first term on the right-hand side of Equation (4.58) defines the desired symbol, whereas 
the remaining series represents the intersymbol interference caused by the timing error At 
in sampling the output y(t). Unfortunately, it is possible for this series to diverge, thereby 
causing erroneous decisions in the receiver. 


e RAISED COSINE SPECTRUM 


We may overcome the practical difficulties encountered with the ideal Nyquist channel by 
extending the bandwidth from the minimum value W = R,/2 to an adjustable value be- 
tween W and 2W. We now specify the overall frequency response P(f) to satisfy a con- 
dition more elaborate than that for the ideal Nyquist channel; specifically, we retain three 
terms of Equation (4.53) and restrict the frequency band of interest to [— W, W], as shown 
by 


1 
P(f) + P(f — 2W) + P(f + 2W) = Iw’ —-Wsf=w (4.59) 
We may devise several band-limited functions that satisfy Equation (4.59), A particular 
form of P(f) that embodies many desirable features is provided by a raised cosine spectrum. 
This frequency response consists of a flat portion and a rolloff portion that has a sinusoidal 


form, as follows: 





1 

IW’ O=|fl<fi 
ye _ [mL fl— w) p o ael 
(f) aw? sin| SUL ; fislf|<2Ww-h 


0, |f| =2W- fı 
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The frequency parameter f, and bandwidth W are related by 


afi 
Ww 


The parameter a is called the rolloff factor; it indicates the excess bandwidth over the 
ideal solution, W. Specifically, the transmission bandwidth By is defined by 


By =2W -f; 
= W(1 + a) 


The frequency response P(f), normalized by multiplying it by 2 W, is plotted in Figure 
4.10a for three values of a, namely, 0, 0.5, and 1. We see that for a = 0.5 or 1, the 


(4.61) 


a=1 





ais, 


-2 -1 


niw 
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FIGURE 4.10 Responses for different rolloff factors. (a) Frequency response. (b) Time response. 
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function P(f) cuts off gradually as compared with the ideal Nyquist channel (i.e., œ = g 
and is therefore easier to implement in practice. Also the function P(f) exhibits odd sym. 
metry with respect to the Nyquist bandwidth W, making it possible to satisfy the condition 
of Equation (4.59). 

The time response p(t) is the inverse Fourier transform of the frequency response 
P(f). Hence, using the P(f) defined in Equation (4.60), we obtain the result (see Problem 
4.13) 


cos(277a Wt) ) re 
62) 


pit) = inewn (a 


which is plotted in Figure 4.100 for a = 0, 0.5, and 1. 

The time response p(t) consists of the product of two factors: the factor sinc(2 Wy 
characterizing the ideal Nyquist channel and a second factor that decreases as 1/|t|? for 
large |t|. The first factor ensures zero crossings of p(t) at the desired sampling instants of 
time ¢ = iT with i an integer (positive and negative). The second factor reduces the tails 
of the pulse considerably below that obtained from the ideal Nyquist channel, so that the 
transmission of binary waves using such pulses is relatively insensitive to sampling time 
errors. In fact, for œ = 1 we have the most gradual rolloff in that the amplitudes of the 
oscillatory tails of p(t) are smallest. Thus the amount of intersymbol interference resulting 
from timing error decreases as the rolloff factor æ is increased from zero to unity. 

The special case with a = 1 (i.e., fı = 0) is known as the full-cosine rolloff charac- 
teristic, for which the frequency response of Equation (4.60) simplifies to 


1 af 
pif) = 14W E + cos( 22) |, 0<|f|<2W (4.63) 
0, |f| =2W 


Correspondingly, the time response p(t) simplifies to 


sinc(4Wt) 


1-7 ewe oe 


This time response exhibits two interesting properties: 


1. Att = +T,/2 = +1/4W, we have p(t) = 0.5; that is, the pulse width measured at 
half amplitude is exactly equal to the bit duration T,. 

2. There are zero crossings at £ = +3T,/2, +5T,/2, ++ * in addition to the usual zero 
crossings at the sampling times ¢ = +T,, +2T;,°--. 


These two properties are extremely useful in extracting a timing signal from the received 
signal for the purpose of synchronization. However, the price paid for this desirable prop- 
erty is the use of a channel bandwidth double that required for the ideal Nyquist channel 
corresponding to a = 0. 


P EXAMPLE 4.2 Bandwidth Requirement of the Ti System 


In Example 3.2 of Chapter 3, we described the signal format for the T1 carrier system thatis 
used to multiplex 24 independent voice inputs, based on an 8-bit PCM word. It was shown 
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that the bit duration of the resulting time-division multiplexed signal (including a framing bit) 
is - 


T, = 0.647 us 


Assuming the use of an ideal Nyquist channel, it follows that the minimum transmission 
bandwidth By of the T1 system is {for a = 0) 


1 
Br = W = — = 772 kHz 
3: a 
However, a more realistic value for the necessary transmission bandwidth is obtained by using 
a full-cosine rolloff characteristic with a = 1. In this case, we find that 


1 
-= +a) = =>-=1. 
By = Wi1+ a) =2W T, 1.544 MHz 4 


l 4.6 Correlative-Level Coding 


Thus far we have treated intersymbol interference as an undesirable phenomenon that 
produces a degradation in system performance. Indeed, its very name connotes a nuisance 
effect. Nevertheless, by adding intersymbol interference to the transmitted signal in a con- 
trolled manner, it is possible to achieve a signaling rate equal to the Nyquist rate of 2W 
symbols per second in a channel of bandwidth W Hertz. Such schemes are called correl- 
ative-level coding or partial-response signaling schemes.’ The design of these schemes is 
based on the following premise: Since intersymbol interference introduced into the trans- 
mitted signal is known, its effect can be interpreted at the receiver in a deterministic way. 
Thus correlative-level coding may be regarded as a practical method of achieving the 
theoretical maximum signaling rate of 2W symbols per second in a bandwidth of W Hertz, 
as postulated by Nyquist, using realizable and perturbation-tolerant filters. 


& DUOBINARY SIGNALING 


The basic idea of correlative-level coding will now be illustrated by considering the specific 
example of duobinary signaling, where “duo” implies doubling of the transmission ca- 
Pacity of a straight binary system. This particular form of correlative-level coding is also 
called class I partial response. 

Consider a binary input sequence {b4} consisting of uncorrelated binary symbols 1 
and 0, each having duration T,. As before, this sequence is applied to a pulse-amplitude 
modulator producing a two-level sequence of short pulses (approximating a unit impulse), 
whose amplitude a, is defined by 


gE i if symbol b; R 1 (4.65) 
-1 if symbol b, is 0 

When this sequence is applied to a duobinary encoder, it is converted into a three- 

level output, namely, —2, 0, and +2. To produce this transformation, we may use the 

scheme shown in Figure 4.11. The two-level sequence {4,} is first passed through a simple 

filter involving a single delay element and summer. For every unit impulse applied to the 
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Figure 4.11 Duobinary signaling scheme. 


input of this filter, we get two unit impulses spaced T, seconds apart at the filter output, 
We may therefore express the duobinary coder output cz as the sum of the present input 
pulse a, and its previous value a,-1, as shown by 


Ce = ay + Api (4.66) 


One of the effects of the transformation described by Equation (4.66) is to change the 
input sequence {a;,} of uncorrelated two-level pulses into a sequence {cg} of correlated three- 


` level pulses. This correlation between the adjacent pulses may be viewed as introducing 


intersymbol interference into the transmitted signal in an artificial manner. However, the 
intersymbol interference so introduced is under the designer’s control, which is the basis 
of correlative coding. 

An ideal delay element, producing a delay of T, seconds, has the frequency response 
exp(—j27fT>), so that the frequency response of the simple delay-line filter in Figure 4.11 
is 1 + exp(—j2afT;). Hence, the overall frequency response of this filter connected in 
cascade with an ideal Nyquist channel is 


Hf) = Hyny fM + exp(—j2afT)] 
= Ayiyauis( flexp(j7fTs) + exp(—jafTs)] exp(—j7fTo) (4.67) 
= 2Hyyquis(f) cos(7fTs) exp(-i7fTs) 


where the subscript | in H;(f) indicates the pertinent class of partial response. For an ideal 
Nyquist channel of bandwidth W = 1/2T,, we have (ignoring the scaling factor T) 


1, |f|s1/2T, 


s (4.68) 
0, otherwise 


Ayyquisl f ) = { 


Thus the overall frequency response of the duobinary signaling scheme has the form ofa 
half-cycle cosine function, as shown by 


aie f cos(afT,) expl-jafTa) |f| = 2T, E 
0, otherwise 
for which the magnitude response and phase response are as shown in Figures 4.12a and 
4.12b, respectively. An advantage of this frequency response is that it can be easily af 
proximated, in practice, by virtue of the fact that there is continuity at the band edges. 
From the first line in Equation (4.67) and the definition of Hyyguise(f) in Equation 
(4.68), we find that the impulse response corresponding to the frequency response Hif) 
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FIGURE 4.12 Frequency response of the duobinary conversion filter. (a) Magnitude response. 
(b) Phase response. 


consists of two sinc (Nyquist) pulses that are time-displaced by T; seconds with respect to 

each other, as shown by (except for a scaling factor) 

a sin(at/T,) 4 sin[w(t — T,)/T;] 
wtlT;, a(t — TaT, 

_ sin(mt/T,) ___sin(at/T,) 

©  atlT, m(t — TT, 

_ T? sin(at/T,) 

~ at(T, — t) 


hilt) 


(4.70) 


The impulse response h(t) is plotted in Figure 4.13, where we see that it has only two 
distinguishable values at the sampling instants. The form of hi(t) shown here explains why 
we also refer to this type of correlative coding as partial-response signaling. The response 
to an input pulse is spread over more than one signaling interval; stated in another way, 
the response in any signaling interval is “partial.” Note also that the tails of h(t) decay as 
1/|z|?, which is a faster rate of decay than the 1/ |¢| encountered in the ideal Nyquist 
channel, 

The original two-level sequence {a,} may be detected from the duobinary-coded 
Sequence {c,} by invoking the use of Equation (4.66). Specifically, let â, represent the 
estimate of the original pulse a, as conceived by the receiver at time t = ki,. Then, sub- 
tracting the previous estimate 4,_, from cg, we get 


ay = Ce Âk- (4.71) 


It is apparent that if c, is received without error and if also the previous estimate 4,_; at 
time ¢ = (k — 1)T, corresponds to a correct decision, then the current estimate â, will be 
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FIGURE 4.13 Impulse response of the duobinary conversion filter, 
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correct too. The technique of using a stored estimate of the previous symbol is calleg 
decision feedback. 

We observe that the detection procedure just described is essentially an inverse of the 
operation of the simple delay-line filter at the transmitter. However, a major drawback of 
this detection procedure is that once errors are made, they tend to propagate through the 
output because a decision on the current input a, depends on the correctness of the decision 
made on the previous input 4—1. 

A practical means of avoiding the error-propagation phenomenon is to use precoding 
before the duobinary coding, as shown in Figure 4.14. The precoding operation performed 
on the binary data sequence {b,} converts it into another binary sequence {d,} defined by 


dy = by ® dy (4.72) 
where the symbol @ denotes modulo-two addition of the binary digits b, and d,_;. This 
addition is equivalent to a two-input EXCLUSIVE OR operation, which is performed ag 
follows: 

z PE 1 if either symbol b, or symbol d-ı (but not both) is 1 
p = 


symbol 0 otherwise (4.73) 


The precoded binary sequence {d,} is applied to a pulse-amplitude modulator, producing 
a corresponding two-level sequence of short pulses {a,}, where a, = +1 as before. This 
sequence of short pulses is next applied to the duobinary coder, thereby producing the 
sequence {c,} that is related to {a,} as follows: 

Cy = ag + apy (4.74) 


Note that unlike the linear operation of duobinary coding, the precoding described by 
Equation (4.72) is a nonlinear operation. 
The combined use of Equations (4.72) and (4.74) yields 


na i if data symbol bx is 1 (4.75 
j +2 if data symbol bẹ is 0 ; 


which is illustrated in Example 4.3. From Equation (4.75) we deduce the following decision 
rule for detecting the original binary sequence {bx} from {cy}: 


If|c,|<1, say symbol b; is 1 










(4.76) 
If|c,|>1, say symbol b, is 0 
Input 
binays == SSeS SS SSS as 
sequence ; Modulo-2 adder Output 
Pulse- j utp! 
{b,} arnplitúde Duobinary Sa three-level 
modulator coaer sequence 
Sample at {ey 


t=kTy, 





am 
Cy 
= 
i 
he 














Precoder 


FIGURE 4.14 A precoded duobinary scheme; details of the duobinary coder are given in Figuie 
4.11. 
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FIGURE 4.15 Detector for recovering original binary sequence from the precoded duobinary 
coder output. 


When |c,| = 1, the receiver simply makes a random guess in favor of symbol 1 or 0. 
According to this decision rule, the detector consists of a rectifier, the output of which is 
compared in a decision device to a threshold of 1. A block diagram of the detector is 
shown in Figure 4.15. A useful feature of this detector is that no knowledge of any input 
sample other than the present one is required. Hence, error propagation cannot occur in 
the detector of Figure 4.15. 


> EXAMPLE 4.3 Duobinary Coding with Precoding 


Consider the binary data sequence 0010110. To proceed with the precoding of this sequence, 
which involves feeding the precoder output back to the input, we add an extra bit to the 
precoder output. This extra bit is chosen arbitrarily to be 1. Hence, using Equation (4.73), we 
find that the sequence (d,} at the precoder output is as shown in row 2 of Table 4.1. The polar 
representation of the precoded sequence {d,} is shown in row 3 of Table 4.1. Finally, using 
Equation (4.74), we find that the duobinary coder output has the amplitude levels given in 
row 4 of Table 4.1. 

To detect the original binary sequence, we apply the decision rule of Equation (4.76), 
and so obtain the binary sequence given in row 5 of Table 4.1. This latter result shows that, 
in the absence of noise, the original binary sequence is detected correctly. 4 


m MODIFIED DUOBINARY SIGNALING 


In the duobinary signaling technique the frequency response H(f), and consequently the 
power spectral density of the transmitted pulse, is nonzero at the origin. This is considered 
to be an undesirable feature in some applications, since many communications channels 
cannot transmit a DC component. We may correct for this deficiency by using the class 
IV partial response or modified duobinary technique, which involves a correlation span 
of two binary digits. This special form of correlation is achieved by subtracting amplitude- 
modulated pulses spaced 2T, seconds apart, as indicated in the block diagram of Figure 


TABLE 4.1 Ilustrating Example 4.3 on duobinary coding 


Binary sequence {b,} 0 0 1 0 1 1 0 
Precoded sequence {d,} 1 1 1 0 0 1 0 
Two-level sequence {a,} +1 +1 +1 —1 -1 +1 -1 -1 
Duobinary coder output {cz} +2 +2 Qo -2 0 0 -2 
Binary sequence obtained by 0 0 1 0 1 1 0 


applying decision rule of Eq. (4.76) 
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FIGURE 4.16 Modified duobinary signaling scheme. 


4.16. The precoder involves a delay of 2T; seconds. The output of the modified duobinary 
conversion filter is related to the input two-level sequence {ax} at the pulse-amplitude mod. 
ulator output as follows: 


Ch = Ap ~ Gp-2 (4.77) 


Here, again, we find that a three-level signal is generated. With a, = +1, we find that ¢, 
takes on one of three values: +2, 0, and —2. 
The overall frequency response of the delay-line filter connected in cascade with an 


ideal Nyquist channel, as in Figure 4.16, is given by 


Ay (f) = Hyyauisef)[1 — exp(-i47fT»)) (4.78) 
= ZH yyquist(f)sin(2afT,) exp(—j27fTs) 


where the subscript IV in Hy(f) indicates the pertinent class of partial response and 
Hyyquist(f) is as defined in Equation (4.68). We therefore have an overall frequency re- 
sponse in the form of a half-cycle sine function, as shown by 


2j sin(2mfT,) exp(—j27fT»), |f| = 1/2T, 


4. 
0, elsewhere (4.73) 


Ay(f) = { 


The corresponding magnitude response and phase response of the modified duobinary 
coder are shown in Figures 4.17a and 4.17), respectively. A useful feature of the modified 
duobinary coder is the fact that its output has no DC component. Note also that this 
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FIGURE 4,17 Frequency response of the modified duobinary conversion filter. (2) Magnitude 
response. (b) Phase response. 
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second form of correlative-level coding exhibits the same continuity at the band edges as 
in duobinary signaling. 

From the first line of Equation (4.78) and the definition of Hyyauisl f) in Equation 
(4.68), we find that the impulse response of the modified duobinary coder consists of two 
sinc (Nyquist) pulses that are time-displaced by 2T, seconds with respect to each other, as 
shown by (except for a scaling factor) 
pane sin(wt/T,) — sin[aw(t — 2T,)/T;,)] 

y ntiT, a(t — 2T)/T, 
_ sin(wt/T,) __sin(t/T,) 
_ 2T? sin(at/T,) 
mtl2T, t) 





This impulse response is plotted in Figure 4.18, which shows that it has three distinguish- 
able levels at the sampling instants. Note also that, as with duobinary signaling, the tails 
of hyy(t) for the modified duobinary signaling decay as 1/|t|?. 

To eliminate the possibility of error propagation in the modified duobinary system, 
we use a precoding procedure similar to that used for the duobinary case. Specifically, 
prior to the generation of the modified duobinary signal, a modulo-two logical addition 
is used on signals 2T, seconds apart, as shown by (see the front end of Figure 4.16) 


dy, = by ® de-z 
ane 1 if either symbol b, or symbol dg-2 (but not both) is 1 (4.81) 
symbol 0 otherwise 


where {b,} is the incoming binary data sequence and {d,} is the sequence at the precoder 
output. The precoded sequence {d,} thus produced is then applied to a pulse-amplitude 
modulator and then to the modified duobinary conversion filter. 

In Figure 4.16, the output digit c, equals —2, 0, or +2, assuming that the pulse- 
amplitude modulator uses a polar representation for the precoded sequence {d,}. Also we 
find that the detected digit b, at the receiver output may be extracted from cg by disre- 
garding the polarity of c,. Specifically, we may formulate the following decision rule: 


If |e > 1, say symbol b; is 1 (4.82) 


If|c|<1, say symbol b, is 0 





FiGurE 4.18 Impulse response of the modified duobinary conversion filter. 
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When |c; | = 1, the receiver makes a random guess in favor of symbol 1 or 0. As with the 
duobinary signaling, we may note the following: 


> In the absence of channel noise, the detected binary sequence {ba} is exactly the same 
as the original binary sequence {b,} at the transmitter input. 

> The use of Equation (4.81) requires the addition of two extra bits to the precodeg 
sequence {a,}. The composition of the decoded sequence {6,) using Equation (4.82) 
is invariant to the selection made for these two bits. 


GENERALIZED FoRM OF CORRELATIVE-LEVEL CODING 
(PARTIAL-RESPONSE SIGNALING) 


The duobinary and modified duobinary techniques have correlation spans of 1 binary digit 
and 2 binary digits, respectively. It is a straightforward matter to generalize these two 
techniques to other schemes, which are known collectively as correlative-level coding or 
partial-response signaling schemes. This generalization is shown in Figure 4.19, where 
Hyyquis(f) is defined in Equation (4.68). It involves the use of a tapped-delay-line filter 
with tap-weights wo, Wi, °°, WN-1- Specifically, different classes of partial-response sig- 


Input Ideal Output 
two-level >| channel, . Nee multilevel 
sequence Hyyque'?)| Sample at Sewanee 


{ay} t= kT, leg 
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FIGURE 4.19 Generalized correlative coding scheme. 
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TABLE 4.2 Different classes of partial-response signaling schemes 
referring to Figure 4.19 





Type of Class N Wo w w w3 Wwa Comments 
I 2 1 1 Duobinary coding 
II 3 1 2 1 
m 3 2 1 =f 
IV 3 0 =1 g Modified duobinary coding 
Vv 5 = 0 2 0 —1 





naling schemes may be achieved by using a weighted linear combination of N ideal Nyquist 
(sinc) pulses, as shown by 


N-1 
ht) = 3 w, sine( £ = n) (4.83) 
n=0 b 


An appropriate choice of the tap-weights in Equation (4.83) results in a variety of spectral 
shapes designed to suit individual applications. Table 4.2 presents the specific details of 
five different classes of partial-response signaling schemes. For example, in the duobinary 
case (class I partial response), we have 

Wg = +1 

w= +1 


and w, = 0 for n = 2. In the modified duobinary case (class IV partial response), we have 


wo = +1 
wW,=0 
w, = -1 


and w, = 0 for n = 3. 
The useful characteristics of partial-response signaling schemes may now be sum- 
marized as follows: 


» Binary data transmission over a physical baseband channel can be accomplished at 
a rate close to the Nyquist rate, using realizable filters with gradual cutoff 
characteristics. 

e Different spectral shapes can be produced, appropriate for the application at hand. 


However, these desirable characteristics are achieved at a price: A larger signal-to-noise 
ratio is required to yield the same average probability of symbol error in the presence of 
noise as in the corresponding binary PAM systems because of an increase in the number 
of signal levels used. 


14.7 Baseband M-ary PAM Transmission 


In the baseband binary PAM system of Figure 4.7, the pulse-amplitude modulator pro- 
duces binary pulses, that is, pulses with one of two possible amplitude levels. On the other 


276 


CHAPTER 4 E BASEBAND PULSE TRANSMISSION 


hand, in a baseband M-ary PAM system, the pulse-amplitude modulator produces one of 
M possible amplitude levels with M > 2. This form of pulse modulation is illustrated jy 
Figure 4.20a for the case of a quaternary (M = 4) system and the binary data sequence 
0010110111. The waveform shown in Figure 4.20a is based on the electrical representa. 
tion for each of the four possible dibits (pairs of bits) given in Figure 4.20b. Note that this 
representation is Gray encoded, which means that any dibit in the quaternary alphabe 
differs from an adjacent dibit in a single bit position. 

In an M-ary system, the information source emits a sequence of symbols from ay 
alphabet that consists of M symbols. Each amplitude level at the pulse-amplitude modu. 
lator output corresponds to a distinct symbol, so that there are M distinct amplitude levels 
to be transmitted. Consider then an M-ary PAM system with a signal alphabet that cop. 
tains M equally likely and statistically independent symbols, with the symbol duration 
denoted by T seconds. We refer to 1/T as the signaling rate of the system, which is expressed 
in symbols per second, or bauds. It is informative to relate the signaling rate of this system 
to that of an equivalent binary PAM system for which the value of M is 2 and the successive 
binary symbols 1 and 0 are equally likely and statistically independent, with the duration 
of either symbol denoted by T, seconds. Under the conditions described here, the binary 
PAM system produces information at the rate of 1/T, bits per seconds. We also observe 
that in the case of a quaternary PAM system, for example, the four possible symbols may 
be identified with the dibits 00, 01, 10, and 11. We thus see that each symbol represents 
2 bits of information, and 1 baud is equal to 2 bits per second, We may generalize this 
result by stating that in an M-ary PAM system, 1 baud is equal to log, M bits per second, 
and the symbol duration T of the M-ary PAM system is related to the bit duration T, of 
the equivalent binary PAM system as 


T = T; log, M (4.84) 


Therefore, in a given channel bandwidth, we find that by using an M-ary PAM system, 
we are able to transmit information at a rate that is log, M faster than the corresponding 
binary PAM system. However, to realize the same average probability of symbol error, an 
M-ary PAM system requires more transmitted power. Specifically, we find that for M much 
larger than 2 and an average probability of symbol error small compared to 1, the trans- 
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FIGURE 4,20 Output of a quaternary system. (a) Waveform. (b) Representation of the 4 possible 
dibits, based on Gray encoding. 
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mitted power must be increased by the factor M*/log, M, compared to a binary PAM 
system. 

In a baseband M-ary system, first of all, the sequence of symbols emitted by the 
information source is converted into an M-level PAM pulse train by a pulse-amplitude 
modulator at the transmitter input. Next, as with the binary PAM system, this pulse train 
is shaped by a transmit filter and then transmitted over the communication channel, which 
corrupts the signal waveform with both noise and distortion. The received signal is passed 
through a receive filter and then sampled at an appropriate rate in synchronism with the 
transmitter. Each sample is compared with preset threshold values (also called slicing lev- 
els), and a decision is made as to which symbol was transmitted. We therefore find that 
the designs of the pulse-amplitude modulator and the decision-making device in an M-ary 
PAM are more complex than those in a binary PAM system. Intersymbol interference, 
noise, and imperfect synchronization cause errors to appear at the receiver output. The 
transmit and receive filters are designed to minimize these errors. Procedures used for the 
design of these filters are similar to those discussed in Sections 4.5 and 4.6 for baseband 
binary PAM systems. 


l 4.8 Digital Subscriber Lines 


At this point in our study of baseband data transmission it is rather appropriate that we 
digress from theoretical aspects of the study and consider a fast-growing application: dig- 
ital subscriber lines.* A digital subscriber line (DSL) operates over a local loop (less than 
1.5 km) that provides a direct connection between a user terminal (e.g., computer) and a 
telephone company’s central office (CO), as illustrated in Figure 4.21. Through the CO, a 
DSL user is connected to a broadband backbone data network, which is based on tech- 
nologies such as the asynchronous transfer mode (ATM) and Internet protocol (IP); these 
technologies and related network resources (i.e., optical fibers, SONET) are discussed in 
the Background and Preview chapter. Accordingly, the information-bearing signal is kept 
in the digital domain all the way from the user terminal to an Internet service provider, 
with the signal being switched or routed at regular intervals in the course of its transmission 
through the data network. 

In the interest of an inexpensive implementation, digital subscriber lines use twisted 
pairs configured to provide a high data-rate, full duplex, digital transmission capability, 
(Twisted pairs are also used for ordinary telephonic communication, as discussed in the 
Background and Preview chapter.) To achieve full-duplex, two-wire transmission, we may 
use one of two possible modes of operation: 


1. Time compression (TC) multiplexing, where data transmission in the two opposite 
directions on the common line are separated in time. Specifically, blocks of bits 
of data are sent in bursts in each direction on an alternate basis, as illustrated in 
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FIGURE 4.21 Block diagram depicting the operational environment of digital subscriber lines. 


278 CHAPTER 4 € BASEBAND PULSE TRANSMISSION 


Transmitter Transmitter 
| Common 
{ line k 


— A 
Receiver _] Receiver 

















(a) 






Transmitter 












Echo 7 
canceller 


A Echo 
f a 


Echo-free Echo-free 


Transmitter 
$ signal signal 
(b) 


FIGURE 4.22 Full-duplex operation using (a) time compression multiplexing, and 
(b) echo-cancellation. i 












Receiver 





Figure 4.22a. To account for propagation time across the line, a guard time is inserted 
between individual bursts of data. Accordingly, the line rate is slightly greater than 
twice the data rate. 

Echo-cancellation mode, which supports the simultaneous flow of data along the 
common line in both directions. For this form of transmission to be feasible, each 
transceiver (transmitter/receiver) includes a hybrid for two purposes: the separation 
of the transmitted signal from the received signal and the two-to-four-wire conver- 
sion, as shown in Figure 4.22b. The hybrid, or more precisely, the hybrid trans- 
former, is basically a bridge circuit with three ports (terminal pairs), as depicted in 
Figure 4.23. If the bridge is not perfectly balanced, the transmitter port of the hybrid 
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FIGURE 4.23 Simplified circuit of hybrid transformer. For the bridge to be balanced, the 
reference impedance Zef should equal the line impedance z;. 
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becomes coupled to the receiver port, thereby giving rise to an echo due to leakage 
of the near-end (local) transmitted signal to the near-end (local) receiver. To cancel 
the unwanted echo, each transceiver includes an echo canceller, as shown in Figure 
4.22. Since data can flow through the line simultaneously in both directions, the 
line rate is the same as the data rate. 


From this discussion, it is apparent that the echo-cancellation scheme offers a much 
better data-transmission performance than the time compression multiplexing scheme, but 
at the expense of increased complexity. However, by implementing the entire transceiver 
in a single very large-scale integrated (VLSI) chip, the cost is made affordable despite the 
increased complexity. In North America, the echo-cancellation scheme has been adopted 
as the basis for designing the transceivers. An adaptive implementation of the echo can- 
celler is discussed in Problem 4.31. 

In addition to echo, there are other impairments of the transmission medium that 
need to be considered. The two dominant impairments are intersymbol interference and 
crosstalk, which are discussed in what follows in that order. 

To a first-order approximation, the squared magnitude response of a twisted pair is 
given by 


|H(f)|? = exp(-eVf) (4.85) 
where 
I 
a= T (4.86) 


In Equation (4.85), the frequency f is measured in kHz, k is a physical constant of the 
twisted pair, l is a reference length (e.g., kilometers), and / is the actual length of the 
twisted pair. Equation (4.85) points to a major impairment in the use of a twisted pair for 
baseband data transmission: the gradual falloff in the frequency response, which, in turn, 
gives rise to intersymbol interference. 

Turning next to crosstalk, the primary cause for its occurrence is the capacitive cou- 
pling that exists between adjacent twisted pairs in a cable. Typically, the nearest five to 
seven twisted pairs in the cable cause most of the crosstalk. In any event, two kinds of 
crosstalk can be observed in a receiver of interest: 


1. Near-end crosstalk (NEXT), which is generated by transmitters located at the same 
end of the cable as the receiver, as illustrated in Figure 4.24a. 
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FIGURE 4.24 (a) Near-end crosstalk (NEXT). (b) Far-end crosstalk (FEXT). 
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FIGURE 4.25 Model of twisted-pair channel. 


2. Far-end crosstalk (FEXT), which is generated by transmitters located further away 
from the receiver, as illustrated in Figure 4.24b. 


FEXT naturally suffers the same line loss as the signal, whereas NEXT does not. Accord. 
ingly, in the echo-cancellation scheme of Figure 4.226 where signals travel in both direc. 
tions in the cable, NEXT will be much stronger than FEXT. Henceforth, we ignore the 
effect of FEXT. 

Indeed, near-end crosstalk and intersymbol interference are the two most important 
factors in determining the performance of a digital subscriber loop. Figure 4.25 shows the 
model of a twisted-pair channel dominated by these two impairments. Since all twisted 
pairs are usually transmitting similar signals, we may model the NEXT as a signal with 
the same power spectral density as the transmitted signal passing through a crosstalk 
frequency response Hyex1(f), which is approximated by 


Anext(f) = Bf ss (4.87) 


where £ is a constant of the cable. The interesting point to note from Figure 4.25 is that 
both the transmitted signal and the interfering signal have the same power spectral density; 
they differ from each other merely in their associated frequency responses, as shown in 
Equations (4.85) and (4.87), respectively. When the model described herein is used for 
simulation study, the transmitted signal is represented by a random data sequence, while 
the interference is represented by a Gaussian noise sequence. 


= LINE CODES FOR DIGITAL SUBSCRIBER LINES 


Now that we have identified the major transmission impairments, we may describe the 
desirable features the spectrum of a transmitted signal should exhibit: 


1. The power spectral density of the transmitted signal should be zero at zero frequency, 
since no DC transmission through a hybrid transformer is possible. 
2. The power spectral density of the transmitted signal should be low at high frequencies 
for the following reasons: 
» Transmission attenuation in a twisted pair is most severe at high frequencies. 
> Crosstalk between adjacent twisted pairs increases dramatically at high frequencies 
because of increased capacitive coupling. In this regard, recall that the impedance 
of a capacitor is inversely proportional to frequency. 
To satisfy these desirable properties, we have to be careful in choosing the line code 
that maps the incoming stream of data bits into electrical pulses for transmission 0° 
the line. Various possibilities, each with its own advantages and disadvantages, exist 
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for such a choice. The list of potential candidates for line codes includes the 

following: 

> Manchester code, which is simple and has zero DC component. Its disadvantage is 
the occupation of a large spectrum, which makes it vulnerable to near-end crosstalk 
and intersymbol interference. (The Manchester code was discussed in Section 3.7.) 

> Modified duobinary code, which has zero DC, is moderately spectrally efficient, 
and causes minimal intersymbol interference. However, simulation studies of the 
crosstalk performance of the modified duobinary code have shown that its im- 
munity to near-end crosstalk and intersymbol interference is about 2 to 3 dB 
poorer than that of block codes on worst-case subscriber lines. (The modified 
duobinary code was discussed in Section 4.6.) 

> Bipolar code, in which successive 1s are represented alternately by positive and 
negative but equal levels, and symbol 0 is represented by a zero level. Bipolar 
signaling has zero DC. Computer simulations have shown that its near-end cross- 
talk and intersymbol interference performance is slightly inferior to the modified 
duobinary code on all digital subscriber loops. (The bipolar code, also known as 
the alternate mark inversion (AMI) codes, was discussed in Section 3.7.) 

> 2B1Q code, which stands for two binary digits encoded into one quaternary sym- 
bol. This code is a block code representing a four-level PAM signal, as illustrated 
in Figure 4.20. Assuming that symbols 1 and 0 are equiprobable, the 2B1Q code 
has zero DC on the average. Moreover, among all the line codes considered herein, 
it offers the greatest baud reduction, and the best performance with respect to 
near-end crosstalk and intersymbol interference. 


It is because of the desirable properties of the 2BIQ code compared to the Manchester 
code, modified duobinary code, the bipolar code, and other line codes not mentioned here,” 
that the 2B1Q code has been adopted as the North American standard for digital sub- 
scriber loops. 

Using the 2B1Q as the line code and VLSI implementation of a transceiver that 
incorporates adaptive equalizers and echo cancellers, it is possible to achieve a bit error 
rate of 1077 operating full duplex at 160 kb/s on the vast majority of twisted-pair sub- 
scriber lines. A bit error rate of 1077 with 12 dB noise margin, when 1 percent worst-case 
NEXT is present, is an accepted performance criterion for digital subscriber lines. Noise 
margin is the amount of receiver noise (including uncancelled echo) that can be tolerated 
without exceeding the 1077 error rate. 


@ ASYMMETRIC DIGITAL SUBSCRIBER LINES 


Another important type of DSL is the asymmetric digital subscriber line (ADSL), which is 
a local transmission system designed to simultaneously support three services on a single 
twisted-wire pair: 


1, Data transmission downstream (toward the subscriber) at bit rates of up to 9 Mb/s. 


2. Data transmission upstream (away from the subscriber) at bit rates of up to 1 Mb/s. 
3. Plain old telephone service (POTS). 


The downstream and upstream bit rates depend on the length of the twisted pair used to 
do the transmission. The DSL is said to be “asymmetric” because the downstream bit rate 
is much higher than the upstream bit rate. Analog voice is transmitted at baseband fre- 
quencies and combined with the passband transmissions of downstream and upstream 
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FIGURE 4.26 (a) Illustrating the different band allocations for an FDM-based ADSL system. 
(b) Block diagram of splitter performing the function of a multiplexer or demultiplexer. Note: both 
filters in the splitter are bidirectional filters. 


data using frequency-division multiplexing (FDM). As illustrated in Figure 4.264, the up- 
stream data transmission is placed in a frequency band different from the downstream 
data transmission to avoid crosstalk. Moreover, a guard band is inserted between the POTS 
band and the upstream transmission band. The coexistence of ADSL and POTS signals on 
the local loop is made possible through the use of a pair of splitters; one splitter is placed 
at the CO end of the local loop and the other one is placed at the user end. In functional 
terms, a splitter is divided into two bidirectional filters, as shown in Figure 4.26b: 


> A low-pass filter for the baseband transmission or extraction of voice signals. 
® A high-pass filter for the passband transmission or extraction of ADSL data. 


In effect, the splitter performs the role of a frequency-division multiplexer or demultiplexer, 
depending on the direction of signal transmission. 

The motivation for making DSL asymmetric is to accommodate “video-on-demand.” 
In such applications, a subscriber needs a high-throughput channel to download high- 
bandwidth video data from a central office on demand. In the reverse direction, a much 
lower throughput channel is adequate to send order information as well as real-time con- 
trol commands. For example, an ADSL for Internet providing downstream transmission 
at the DS1 rate of 1.544 Mb/s and upstream transmission of about 160 kb/s would meet 
the requirements of this application. The approximate 10:1 asymmetry ratio realized by 
such a system prevents the flow of acknowledgment packets in the IP from becoming a 
bottleneck to the faster direction of data transmission. 

It is very difficult to transmit data over a twisted pair at the DS1 rate and higher, 
and doing so requires the use of sophisticated modulation techniques. The treatment of 
this subject is deferred to Chapter 6. 


| 4.9 Optimum Linear Receiver 


Resuming our study of the baseband data transmission system depicted in Figure 4,7, we 
have thus far treated the following two channel conditions separately: 
> Channel noise acting alone, which led to formulation of the matched filter receiver. 


> Intersymbol interference acting alone, which led to formulation of the pulse-shapité 
transmit filter so as to realize the Nyquist channel. 
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In a real-life situation, however, channel noise and intersymbol interference act together, 
affecting the behavior of a data transmission system in a combined manner. In this section, 
we formulate the basis for designing a linear receiver optimized for the general case of a 
linear channel that is both dispersive and noisy. 

In one approach to the design of a linear receiver, the receiver is viewed as a zero- 
forcing equalizer followed by a decision-making device. The objective of this form of 
equalization is to have the “intersymbol interference forced to zero” at all the instants 
t = kT at which the channel output is sampled, except for k = 0 where the symbol of 
interest is assumed to occur. Under this condition, symbol-to-symbol detection is assured 
to be optimal in accordance with the Nyquist criterion, provided that the channel noise 
w(t) is zero. 

The zero-forcing equalizer is relatively easy to design because it ignores the effect of 
the channel noise w(t). A serious consequence of this oversight, however, is that it leads 
to overall performance degradation due to noise enhancement, a phenomenon that is an 
inherent feature of zero-forcing equalization; see Problem 4.32. A more refined approach 
for the receiver design is to use the mean-square error criterion, which provides a balanced 
solution to the problem of reducing the effects of both channel noise and intersymbol 
interference. Indeed, for a prescribed computational complexity, an equalizer designed on 
this latter basis always performs as well as, and often better than, its zero-forcing coun- 
terpart. Henceforth, we concentrate on the mean-square error criterion for receiver design. 

Referring back to the baseband binary data transmission system of Figure 4.7, the 
receive filter characterized by the impulse response c(t) produces the following response 
due to the channel output x(t): 


y(t) = E c(7)x(t — 7) dr (4.88) 
The channel output x(t) is itself defined by 
x(t) = >) agit — kT,) + w(t) (4.89) 
k 
where a, is the symbol transmitted at time t = kT, and w(t) is the channel noise. The time 
function q(t) is the convolution of two impulse responses: g(t) pertaining to the pulse- 


shaping transmit filter, and h(t) pertaining to the channel. Substituting Equation (4.89) 
into (4.88) and sampling the resulting output y(t) at time ¢ = 7T,, we may write 


yGT,) = & +n; (4.90) 


where & is the signal component defined by 


Esda A c(r)qliT, — kT, — 7) dr (4.91) 
A pe 


and 7; is the noise component defined by 


n= i e(7r)w(iT, — 7) dr (4.92) 


The condition for perfect operation of the receiver is to have y(iT,) = a; where a; is the 
transmitted symbol. Deviation from this condition results in the error signal 


e: = WIT) — a; 


a (4.93) 
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Accordingly, we may formally define the mean-square error as 
1 
LaF E[e?] (4.94) 


where E is the statistical expectation operator, and the factor 1/2 is introduced for cop, 
venience of presentation. Substituting Equation (4.93) into (4.94) and then expanding 
terms, we get 


J = E EE) + È Eln) +5 Elat) + Elem] — Flea - Elga] (435) 


We now evaluate the six expectation terms in this equation in the order they appear here, 


1. Ina stationary environment the mean-square term E[é?] is independent of the instant 
of time t = iT, at which the receive filter output is sampled. Hence, we may simplify 
the expression of this term by writing 


EII = È D Elaa] E i elr)e(r2)q(IT, — n)alkT, — t) dr, dr, 


Assuming that, first, the binary symbols a, = +1 as in Equation (4.42) and, second, 
the transmitted symbols are statistically independent, that is, 


1 forl=k 
Elaa] = f otherwise (4.96) 
we may further reduce the expression for the mean-square term E[£?] to 
EE = ff Ran mletrdeln) dn dr (4.97 
where 
Ralu 72) = È RTs ~ v)gtkTs — 72) (4.98) 


The factor R,(t1, T2) is the temporal autocorrelation function of the sequence 
{q(kT,)}. Stationarity of this sequence means that (see Section 1.5) 


R(t, n) = R(t -7 = R(t — h) 
2. The mean-square term E[?] due to channel noise is given by (using Equation (4.92)) 
E[nj] = ii ji c(t )e(7)E[w(iT, — 7%)(iT, — %)] dr dr 
a (4.99) 
= ia iz e(n)e(m)Rw(t — n) dt, dr 


where Ry(t2 — 7) is the ensemble-averaged autocorrelation function of the channel 
noise w(t). With w(t) assumed to be white with power spectral density Ny/2, we have 


N 
Rylm — 1) = a5 a(t — 7) (4.100) 


Hence, the expression for E[7?] simplifies to 


ei] =A f dnd) ata- o) dnd (4101) 
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3. The mean-square term E[a;] due to the transmitted symbol 4; is unity by virtue of 
Equation (4.96); that is, 


E[@]=1 for alli (4.102) 


4, The expectation of the cross-product term £7; is zero for two reasons: first é; and 74; 
are independent and, second, the channel noise w(t), and therefore #;, has zero mean; 
that is, 


E[én] =0 for alli (4.103) 

5. For similar reasons, the expectation of the cross-product term 7;q; is also zero; that 
is, 

E[nja] =0 for alli (4.104) 


6. Finally, the expectation of the cross-product term é,a; is given by (using Equation 
(4.91)) 


Elga] = > Elana’ f a qliTe — kT, — 7) dr (4.105) 


By virtue of the statistical pee of the transmitted symbols described in 
Equation (4.96), this expectation reduces to 


Figa] = [dnar dr (4.106) 


Thus substituting Equations (4.97), (4.101) to (4.104) and (4.106) into (4.95), we 
may express the mean-square error J for the binary data transmission system of 
Figure 4.7 as 


+E i (zai (t— 1) + ` S(t — n) e(t)e(r) dt dr — k c(t)qa(—t) dt 


(4.107) 


For convenience of presentation, we have made the following changes in variables: 
7, and 7 in the first integral are replaced by t and 7, respectively, and 7 is replaced 
with ¢ in the second integral. Note also that this expression for the mean-square error 
J is in actual fact normalized with respect to the variance of the transmitted symbols 
a, by virtue of the assumption made in Equation (4.96). 

With the formula of Equation (4.107) for the mean-square error J at hand, we 
are now ready to specify the design of the receive filter in Figure 4.7. Differentiating 
Equation (4.107) with respect to the impulse response c(t) of the receive filter, and 
then setting the result equal to zero, we get 


Í (ra =n) ck = a(t — n)a dr = q(-t) (4.108) 
Equation (4.108) is the formula for finding the impulse response c(t) of the equalizer 
optimized in the mean-square error sense. An equalizer so designed is referred to as 


the minimum-mean square error (mmse) equalizer. 
Taking the Fourier transform of both sides of Equation (4.108), we obtain 


(sun + Soi =o") (4.109) 
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where c(t) = C(f), q(t) = O(f), and Rg = S,(f). Solving Equation (4.109) for Cif) 
we get x 3 


af) = = (4.119) 


In Problem 4.33 it is shown that the power spectral density of the sequence [q(kT,)) 


can be expressed as 
k 
or) 


which means that the frequency response C(f) of the optimum linear receiver js 

periodic with period 1/T,. Equation (4.110) suggests the interpretation of the opti. 

mum linear receiver as the cascade connection of two basic components:* 

» A matched filter whose impulse response is q(—t), where q(t) = g(t) x h(t). 

> A transversal (tapped-delay-line) equalizer whose frequency response is the inverse 
of the periodic function S,(f) + (No/2). 


2 








1 
Saf) = T 2 (4.111) 


To implement Equation (4.110) exactly we need an equalizer of infinite length. In practice, 
we may approximate the optimum solution by using an equalizer with a finite set of 
coefficients {c,}8__n, provided N is large enough. Thus the receiver takes the form shown 
in Figure 4.27. Note that the block labeled z7) in Figure 4.27 introduces a delay equal to 
Tp, which means that the tap spacing of the equalizer is exactly the same as the bit duration 
T,. An equalizer so configured is said to be synchronous with the transmitter. 


@ PRACTICAL CONSIDERATIONS 


The mmse receiver of Figure 4.27 works well in the laboratory, where we have access to 
the system to be equalized, in which case we may determine a transversal equalizer char- 
acterized by the set of coefficients {c} -n which provides an adequate approximation 
to the frequency response C(f) of Equation (4.110): In a real-life telecommunications 
environment, however, the channel is usually time varying. For example, in a public 
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FIGURE 4.27 Optimum linear receiver consisting of the cascade connection of matched filter 
and transversal equalizer. 
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switched telephone network, we find that two factors contribute to the distribution of 
pulse distortion on different link connections: 


> Differences in the transmission characteristics of the individual links that may be 
switched together. 
® Differences in the number of links in a connection. 


The result is that the telephone channel is random in the sense of being one of an ensemble 
of possible physical realizations. Consequently, the use of a fixed pair of matched filter 
and equalizer designed on the basis of average channel characteristics may not adequately 
reduce the effects of intersymbol interference and channel noise. To realize the full trans- 
mission capability of the telephone channel, we need an adaptive receiver? that provides 
for the adaptive implementation of both the matched filter and the equalizer in a combined 
manner. The receiver is adaptive in the sense that the equalizer coefficients are adjusted 
automatically in accordance with a built-in algorithm. 

Another point of interest is that it may be desirable to have the taps of the equalizer 
spaced by an amount closer than the symbol period; typically, the spacing between adjacent 
taps is set equal to T/2. The resulting structure is known as a fractionally spaced equalizer 
(FSE). An FSE has the capability of compensating for delay distortion much more effec- 
tively than a conventional synchronous equalizer. Another advantage of the FSE is the fact 
that data transmission may begin with an arbitrary sampling phase. However, mathemat- 
ical analysis of the FSE is more complicated than for a synchronous equalizer and will 
therefore not be pursued here.'° 


| 4.10 Adaptive Equalization 


In this section we develop a simple and yet effective algorithm for the adaptive equalization 
of a linear channel of unknown characteristics. Figure 4.28 shows the structure of an 
adaptive synchronous equalizer, which incorporates the matched filtering action. The al- 
gorithm used to adjust the equalizer coefficients assumes the availability of a desired re- 
sponse. One’s first reaction to the availability of a replica of the transmitted signal is: If 
such a signal is available at the receiver, why do we need adaptive equalization? To answer 
this question, we first note that a typical telephone channel changes little during an average 
data call. Accordingly, prior to data transmission, the equalizer is adjusted under the guid- 
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FIGURE 4.28 Block diagram of adaptive equalizer. 
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ance of a training sequence transmitted through the channel. A synchronized version of 
this training sequence is generated at the receiver, where (after a time shift equal to the 
transmission delay through the channel) it is applied to the equalizer as the desired re. 
sponse. A training sequence commonly used in practice is the pseudonoise (PN) sequence 
which consists of a deterministic periodic sequence with noise-like characteristics, Two 
identical PN sequence generators are used, one at the transmitter and the other at the 
receiver. When the training process is completed, the PN sequence generator is switched 
off, and the adaptive equalizer is ready for normal data transmission. Detailed description 
of PN sequence generators is presented in Chapter 7. 


LEAST-MEAN-SQUARE ALGORITHM (REVISITED) 
To simplify notational matters, we let 

x[n] = x(nT) 

yla] = (nT) 


Then, the output y[7] of the tapped-delay-line equalizer in response to the input sequence 
{x[n]} is defined by the discrete convolution sum (see Figure 4.28) 


N 
yin} = > w,x{n — k] (4.112) 


where wp is the weight at the kth tap, and N + 1 is the total number of taps. The tap- 
weights constitute the adaptive filter coefficients. We assume that the input sequence {x[n}} 
has finite energy. We have used a notation for the equalizer weights in Figure 4.28 thats 
different from the corresponding notation in Figure 4.27 to emphasize the fact that the 
equalizer in Figure 4.28 also incorporates matched filtering. 

The adaptation may be achieved by observing the error between the desired pulse 
shape and the actual pulse shape at the filter output, measured at the sampling instant, 
and then using this error to estimate the direction in which the tap-weights of the filter 
should be changed so as to approach an optimum set of values. For the adaptation, we 
may use a criterion based on minimizing the peak distortion, defined as the worst-case 
intersymbol interference at the output of the equalizer. The development of an adaptive 
equalizer using such a criterion builds on the zero-forcing concept described briefly in 
Section 4.9. However, the equalizer is optimum only when the peak distortion at its input 
is less than 100 percent (i.e., the intersymbol interference is not too severe). A better 
approach is to use a mean-square error criterion, which is more general in application; 
also an adaptive equalizer based on the mean-square error criterion appears to be less 
sensitive to timing perturbations than one based on the peak distortion criterion. Accord- 
ingly, in what follows we use the mean-square error criterion to derive the adaptive equal- 
ization algorithm. 

Let ajn] denote the desired response defined as the polar representation of the nib 
transmitted binary symbol. Let e[n] denote the error signal defined as the difference be- 
tween the desired response a[n] and the actual response y[7] of the equalizer, as shown by 


elz] = a[n] — yla] (4.113) 


In the least-mean-square (LMS) algorithm" for adaptive equalization, the error signal ef] 
actuates the adjustments applied to the individual tap weights of the equalizer as th 
algorithm proceeds from one iteration to the next. A derivation of the LMS algorithm fot 
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adaptive prediction was presented in Section 3.13. Recasting Equation (3.72) into its most 
general form, we may state the formula for the LMS algorithm in words as follows: 


Input signal 
[otis | í Old ar . ( Siepie ) | applied to (Eror 


of kth tap- of kth tap- parameter kth tap- a (4.114) 


i igh 
weight weight elie 


Let u denote the step-size parameter. From Figure 4.28 we see that the input signal applied 
to the kth tap-weight at time step # is x[” — k]. Hence, using ,(n) as the old value of 
the kth tap-weight at time step #, the updated value of this tap-weight at time step n + 1 
is, in light of Equation (4.114), defined by 


faln + 1] = dln] + pxln — klefa], & =0,1,...,N (4.115) 


where 
N 


eln] = aln] — & wd, [n]x[n — k] (4.116) 


k=0 


These two equations constitute the LMS algorithm for adaptive equalization. Note that 
the length of the adaptive equalizer in Figure 4.28 is not to be confused with the length of 
the equalizer in Figure 4.27. 

We may simplify the formulation of the LMS algorithm using matrix notation. Let 
the (N + 1)-by-1 vector x[n] denote the tap-inputs of the equalizer: 


x[a] = [x[n],...,x[/2 — N + 1], x[n - NJ] (4.117) 
where the superscript T denotes matrix transposition. Correspondingly, let the 
(N + 1)-by-1 vector Ŵ[n] denote the tap-weights of the equalizer: 

win] = bola], ĉin], -Ên (4.118) 


We may then use matrix notation to recast the convolution sum of Equation (4.112) in 
the compact form 


yia] = xin] sin] (4.119) 


where x’[n]W[1] is referred to as the inner product of the vectors x[n] and Ŵ[n]. We may 
now summarize the LMS algorithm for adaptive equalization as follows: 


1. Initialize the algorithm by setting W[1] = 0 (i.e., set all the tap-weights of the equalizer 
to zero at # = 1, which corresponds to time t = T). 
2. Forn = 1, 2, . . . , compute 
yia] = x"[n]W[n] 
eln] = aln] — yi7] 
W[n + 1] = fn] + ueln]xia] 
where w is the step-size parameter. 
3. Continue the iterative computation until the equalizer reaches a “‘steady state,” by 


which we mean that the actual mean-square error of the equalizer essentially reaches 
a constant value. 


The LMS algorithm is an example of a feedback system, as illustrated in the block 
diagram of Figure 4.29, which pertains to the kth filter coefficient. It is therefore possible 
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FIGURE 4.29 Signal-flow graph representation of the LMS algorithm involving the kth tap 
weight. 


for the algorithm to diverge (i.e., for the adaptive equalizer to become unstable). Unfor. 
tunately, the convergence behavior of the LMS algorithm is difficult to analyze. Neverthe. 
less, provided that the step-size parameter yu is assigned a small value, we find that after a 
large number of iterations the behavior of the LMS algorithm is roughly similar to that of 
the steepest-descent algorithm, which uses the actual gradient rather than a noisy estimate 
for the computation of the tap-weights. (The steepest-descent algorithm was discussed in 
Section 3.13.) 


E OPERATION OF THE EQUALIZER 


There are two modes of operation for an adaptive equalizer, namely, the training mode 
and decision-directed mode, as shown in Figure 4.30. During the training mode, as ex- 
plained previously, a known PN sequence is transmitted and a synchronized version of it 
is generated in the receiver, where (after a time shift equal to the transmission delay) it is 
applied to the adaptive equalizer as the desired response; the tap-weights of the equalizer 
are thereby adjusted in accordance with the LMS algorithm. 

When the training process is completed, the adaptive equalizer is switched to its 
second mode of operation: the decision-directed mode. In this mode of operation, the error 


signal is defined by 






ela] = 4[n] — yir] (4.120) 
Adaptive Decision 
equalizer device 











Training 
sequence 
generator 








FIGURE 4.30 Illustrating the two operating modes of an adaptive equalizer: For the training 
mode, the switch is in position 1; and for the tracking mode, it is moved to position 2. 
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where y(7] is the equalizer output at time t = nT, and 4[] is the final (not necessarily) 
correct estimate of the transmitted symbol a[n]. Now, in normal operation the decisions 
made by the receiver are correct with high probability. This means that the error estimates 
are correct most of the time, thereby permitting the adaptive equalizer to operate satisfac- 
torily. Furthermore, an adaptive equalizer operating in a decision-directed mode is able to 
track telatively slow variations in channel characteristics. 

It turns out that the larger the step-size parameter p, the faster the tracking capability 
of the adaptive equalizer. However, a large step-size parameter jz may result in an unac- 
ceptably high excess mean-square error, defined as that part of the mean-square value of 
the error signal in excess of the minimum attainable value Juin (which results when the 
tap-weights are at their optimum settings). We therefore find that in practice the choice of 
a suitable value for the step-size parameter u involves making a compromise between fast 
tracking and reducing the excess mean-square error. 


8 DECISION-FEEDBACK EQUALIZATION 


To develop further insight into adaptive equalization, consider a baseband channel with 
impulse response denoted in its sampled form by the sequence {/[7]} where h[n] = h(nT). 
The response of this channel to an input sequence {x[7]}, in the absence of noise, is given 
by the discrete convolution sum 


yin] = > biklx[n — k] 
f (4.121) 


= b[0]x[n] + È hik]x[n — k] + X, hik]x[n — k] 
<0 k>0 


The first term of Equation (4.121) represents the desired data symbol. The second term is 
due to the precursors of the channel impulse response that occur before the main sample 
[0] associated with the desired data symbol. The third term is due to the postcursors of 
the channel impulse response that occur after the main sample [0]. The precursors and 
postcursors of a channel impluse response are illustrated in Figure 4.31. The idea of de- 
cision-feedback equalization’? is to use data decisions made on the basis of precursors of 
the channel impulse response to take care of the postcursors; for the idea to work, however, 
the decisions would obviously have to be correct. Provided that this condition is satisfied, 


h10] 





Precursors Postcursors 


FIGURE 4.31 Impulse response of a discrete-time channel, depicting the precursors and 
postcursors, 
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FIGURE 4.32 Block diagram of decision-feedback equalizer. 


a decision-feedback equalizer is able to provide an improvement over the performance of 
the tapped-delay-line equalizer. 

A decision-feedback equalizer (DFE) consists of a feedforward section, a feedback 
section, and a decision device connected together as shown in Figure 4.32. The feedforward 
section consists of a tapped-delay-line filter whose taps are spaced at the reciprocal of the 
signaling rate. The data sequence to be equalized is applied to this section. The feedback 
section consists of another tapped-delay-line filter whose taps are also spaced at the recip- 
rocal of the signaling rate. The input applied to the feedback section consists of the deci- 
sions made on previously detected symbols of the input sequence. The function of the 
feedback section is to subtract out that portion of the intersymbol interference produced 
by previously detected symbols from the estimates of future samples. 

Note that the inclusion of the decision device in the feedback loop makes the equal- 
izer intrinsically nonlinear and therefore more difficult to analyze than an ordinary tapped- 
delay-line equalizer. Nevertheless, the mean-square error criterion can be used to obtain a 
mathematically tractable optimization of a decision-feedback equalizer. Indeed, the LMS 
algorithm can be used to jointly adapt both the feedforward tap-weights and the feedback 
tap-weights based on a common error signal; see Problem 4.37. 

On the basis of extensive comparative evaluations of a linear equalizer and decision- 
feedback equalizer reported in the literature,” we may report that when the frequency 
response of a linear channel is characterized by severe amplitude distortion or relatively 
sharp amplitude cutoff, the decision-feedback equalizer offers a significant improvement 
in performance over a linear equalizer for an equal number of taps. It is presupposed here 
that the feedback decisions in the DFE are all correct. For an example of sharp amplitude 
cutoff, see the frequency response of a telephone channel depicted in Figure 8 in the Back- 
ground and Preview chapter. 

Unlike a linear equalizer, a decision-feedback equalizer suffers from error propage- 
tion. However, despite the fact that the DFE is a feedback system, error propagation wil 
not persist indefinitely. Rather, decision errors tend to occur in bursts. To justify this kind 
of behavior, we offer the following intuitive reasoning:* 


> Let L denote the number of taps in the feedback section of a DFE. After a sequence 
of L consecutive correct decisions, all decision errors in the feedback section will be 
flushed out. This points to an error propagation of finite duration. 

e When a decision error is made, the probability of the next decision being erroneous 
too is clearly no worse than 1/2. 

> Let K denote the duration of error propagation, that is, the number of symbols 
needed to make L consecutive correct decisions. Then the average error rate 15 
(K/2)Po, where K/2 is the average number of errors produced by a single decision 
error, and Pp is the probability of error given that the past L decisions are all correct 
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> In a fair-coin tossing experiment, the average number of coin tosses, K, needed to 
get L successive heads (representing no errors) turns out to be 2(2* — 1). 


It follows therefore that the effect of error propagation in a decision-feedback equalizer is 
to increase the average error rate by a factor approximately equal to 2”, compared to the 
probability of making the first error. For example, for L = 3 the average error rate is 
increased by less than an order of magnitude due to error propagation. 


l 4.11 Computer Experiments: Eye Patterns 


In previous sections of this chapter we have discussed various techniques for dealing with 
the effects of channel noise and intersymbol interference on the performance of a baseband 
pulse-transmission system. In the final analysis, what really matters is how to evaluate the 
combined effect of these impairments on overall system performance in an operational 
environment. An experimental tool for such an evaluation in an insightful manner is the 
so-called eye pattern, which is defined as the synchronized superposition of all possible 
realizations of the signal of interest (e.g., received signal, receiver output) viewed within a 
particular signaling interval. The eye pattern derives its name from the fact that it resembles 
the human eye for binary waves. The interior region of the eye pattern is called the eye 
opening. 

An eye pattern provides a great deal of useful information about the performance of 
a data transmission system, as described in Figure 4.33. Specifically, we make the following 
statements: 


> The width of the eye opening defines the time interval over which the received signal 
can be sampled without error from intersymbol interference; it is apparent that the 
preferred time for sampling is the instant of time at which the eye is open the widest. 

> The sensitivity of the system to timing errors is determined by the rate of closure of 
the eye as the sampling time is varied. 

> The height of the eye opening, at a specified sampling time, defines the noise margin 
of the system. 
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FIGURE 4.33 Interpretation of the eye pattern. 
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When the effect of intersymbol interference is severe, traces from the upper portion of the 
eye pattern cross traces from the lower portion, with the result that the eye is completely 
closed. In such a situation, it is impossible to avoid errors due to the combined presence 
of intersymbol interference and noise in the system. 

In the case of an M-ary system, the eye pattern contains (M — 1) eye openings stacked 
up vertically one on the other, where M is the number of discrete amplitude levels used to 
construct the transmitted signal. In a strictly linear system with truly random data, all these 
eye openings would be identical. 

In the next two experiments, we use computer simulations to study the eye patterns 
for a quaternary (M = 4) baseband PAM transmission system under noiseless, noisy, and 
band-limited conditions. The effect of channel nonlinearity on eye patterns is discussed in 
Problem 4.38. 


Experiment 1: Effect of Channel Noise 


Figure 4.34a shows the eye diagram of the system under idealized conditions: no channel 
noise and no bandwidth limitation. The source symbols used are randomly generated on 
a computer, with raised cosine pulse-shaping. The system parameters used for the gener- 
ation of the eye diagram are as follows: Nyquist bandwidth W = 0.5 Hz, rolloff factor 
a = 0.5, and symbol duration T = T, log, M = 2T,. The openings in Figure 4.34 are 
perfect, indicating reliable operation of the system. Note that this figure has M — 1 = 3 
openings. 

Figures 4.346 and 4.34c show the eye diagrams for the system, but this time with 
channel noise corrupting the received signal. These two figures were simulated for signal- 
to-noise ratio SNR = 20 dB and 10 dB, respectively, with the SNR being measured at the 
channel output. When SNR = 20 dB the effect of channel noise is hardly discernible in 
Figure 4.346, but when SNR = 10 dB the openings of the eye diagram in Figure 4.34c are 
barely visible. 


Experiment 2: Effect of Bandwidth Limitation 


Figures 4.354 and 4.35 b show the eye diagrams for the quaternary system using the same 
parameters as before, but this time under a bandwidth-limited condition and a noiseless 
channel. Specifically, the channel is now modeled by a low-pass Butterworth filter, whose 
squared magnitude response is defined by 


1 
HAI = EF 
where N is the order of the filter, and fo is its 3-dB cutoff frequency. For the computer 
experiment described in Figure 4.35a, the following values are used: 
N = 25 and fo = 0.975 Hz 
The bandwidth required by the PAM trasmission system is computed to be 
By = W(1 + a) = 0.75 Hz 


Although the channel bandwidth (i.e., cutoff frequency) is greater than absolutely neces- 
sary, its effect on the passband is observed as a decrease in the size of the eye openings 
compared to those in Figure 4.34a. Instead of the distinct values at time ¢ = 1 s (as show" 
in Figure 4.34a), now there is a blurred region. 


{a) 


Amplitude 





0 02 04 06 08 1 #12 #14 #16 #18 2 


Normalized time t/T, 


(b) 


0.5 


Amplitude 
Q 






0 02 04 06 08 1 12 14 16 18 
Normalized time z/T, 


(e) 


Amplitude 


b 





0 02 04 06 08 1 12 14 #16 18 2 
Normalized time t/T, 


Figure 4.34 (a) Eye diagram for noiseless quaternary system. (b) Eye diagram for quaternary 
system with SNR = 20 dB. (c) Eye diagram for quaternary system with SNR = 10 dB. 
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FIGURE 4.35 (a) Eye diagram for noiseless band-limited quaternary system: cutoff frequency 
fo = 0.975 Hz. (b) Eye diagram for noiseless band-limited quaternary system: cutoff frequency 


fo = 0.5 Hz. 


In Figure 4.350 the channel bandwidth is reduced further by modeling the channel 
as a low-pass Butterworth filter with N = 25 and fy = 0.5 Hz. The effect of reduced 
channel bandwidth is to further reduce the extent to which the eyes are open. 


| 4.12 Summary and Discussion 


In this chapter, we studied the effects of channel noise and intersymbol interference on the 
performance of baseband-pulse transmission systems. Intersymbol interference (ISI) is dif- 
ferent from noise in that it is a signal-dependent form of interference that arises because 
of deviations in the frequency response of a channel from the ideal low-pass filter (Nyquist 
channel); it disappears when the transmitted signal is switched off. The result of these 
deviations is that the received pulse corresponding to a particular data symbol is affected 
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by the tail ends of the pulses representing the previous symbols and the front ends of the 
pulses representing the subsequent symbols. 

Depending on the received signal-to-noise ratio, we may distinguish three different 
situations that can arise in baseband-pulse transmission systems for channels with fixed 
characteristics: 


1. The effect of ISI is negligible in comparison to that of channel noise. 
The proper procedure in this case is to use a matched filter, which is the optimum linear 
time-invariant filter for maximizing the peak pulse signal-to-noise ratio. 


2. The received signal-to-noise ratio is high enough to ignore the effect of channel noise. 
In this case, we need to guard against the effects of ISI on the reconstruction of the trans- 
mitted data at the receiver. In particular, control must be exercised over the shape of the 
received pulse. This design objective can be achieved in one of two different ways: 


e Using a raised cosine spectrum for the overall frequency response of the baseband- 
pulse transmission system. 

» Using correlative-level coding or partial-response signaling that adds ISI to the trans- 
mitted signal in a controlled manner. 


3. The ISI and noise are both significant. 

For a mathematically tractable solution to this more difficult situation, we may use the 
mean-square error criterion. The resulting optimum linear receiver is called the minimum 
mean-square error (mmse) receiver. It consists of the cascade connection of a matched filter 
and linear transversal (tapped-delay-line) equalizer. 


When, however, the channel is random in the sense of being one of an ensemble of 
possible physical realizations, which is frequently the case in a telecommunications envi- 
ronment, the use of fixed filter designs based on average channel characteristics may not 
be adequate. In situations of this kind, the preferred approach is to use an adaptive equal- 
izer, the purpose of which is to compensate for variations in the frequency response of the 
channel automatically during the course of data transmission. The combined use of a 
tapped-delay-line filter and the least-mean-square (LMS) algorithm for adjusting the tap- 
weights provides the basis of a simple and yet highly effective method for implementing 
the adaptive equalizer. Such a device is capable of dealing with the combined effects of ISI 
and receiver noise in a nonstationary environment. Its practical value lies in the fact that 
almost every modem (modulator-demodulator) in commercial use today for the transmis- 
sion of digital data over a voice-grade telephone channel uses an adaptive equalizer as an 
integral part. 

Another important application of adaptive filtering is in the design of echo cancellers 
that constitute a critical component of transceivers for digital subscriber lines. Typically, 
a digital subscriber line uses a twisted pair as the transmission medium, the very same one 
used in ordinary telephone channels. However, unlike telephone channels, digital sub- 
scriber lines are designed to provide a high data-rate digital transmission capability be- 
tween a digital network and subscriber plants, with a data rate of 64 kb/s and up. 


| NOTES AND REFERENCES 


1. The classic books on baseband-pulse transmission are Lucky, Salz, and Weldon (1968) and 
Sunde (1969). For detailed treatment of different aspects of the subject, see Gitlin, Hayes, 
and Weinstein (1992). Proakis (1995), and Benedetto, Biglieri, and Castellani (1987). 
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2. The characterization of a matched filter was first derived by North in a classified r a 


(RCA Laboratories Report PTR-6C, June 1943), which was published 20 years later: sẹ 
the paper by North (1963). A similar result was obtained independently by Van Vleck a 
Middleton, who coined the term matched filter: see the paper by Van Vleck and Middleton 
(1946). For review material on the matched filter and its properties, see the papers by Turn 
(1960, 1976). 


. The error function denoted by erf(u), is defined in a number of different ways in the liter. 


ature. We shall use the following definition: 
erf(u) = 4f exp(—z?) dz 


The error function has two useful properties: 


(i) erf(—u) = —erf(x) 
This is known as the symmetry relation. 
(ii) As u approaches infinity, erf(«) approaches unity; that is, 


4f exp(-z’) dz = 1 
7 


The complementary error function is defined by 
erfc(u) = 4f exp(—z°} dz 


which is related to the error function as follows: 
erfc(u) = 1 — erf(u) 


Table A6.6 gives values of the error function erf{u) for u in the range 0 to 3.3. 
For large positive values of u, we have two simple bounds on erfc(u), one lower and 
the other upper, as shown by 


expl) f, 1 exp(—u?) 
Vau (: i) < erfol) < — ru 


The complementary error function provides the basis for a compact formulation of 
the probability of symbol error, as explained in Section 4.3. Another function that is also 
commonly used in the literature for this purpose is the Q-function. Consider a standardized 
Gaussian random variable X of zero:mean and unit variance. The probability that an 
observed value of the random variable X will be greater than vis given by the O-function: 


cS 2 
Qh) = sl, exp(-%) dx 


The Q-function defines the area under the standardized Gaussian tail. The Q-function is 
related to the complementary error function as 


1 v 
Ql) = 3 ere} 


Conversely, putting u = v2, we have 
erfe(u) = 2O(V2u) 


. The criterion described in Equation (4.49) or Equation (4.53) was first formulated by 


Nyquist in the study of telegraph transmission theory; the 1928 paper by Nyquist 15 ĉ 
classic. In the literature, this criterion is referred to as Nyquist’s first criterion. In his 1928 
paper, Nyquist described another method, referred to in the literature as Nyquist’s secon 
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criterion. The second method makes use of the instants of transition between unlike sym- 
bols in the received signal rather than centered samples. A discussion of the first and second 
criteria is presented in Bennett (1970, pp. 78-92) and in the paper by Gibby and Smith 
(1965). A third criterion attributed to Nyquist is discussed in Sunde (1969); see also the 
papers by Pasupathy (1974) and Sayar and Pasupathy (1987). 


. Correlative-level coding and partial-response signaling are synonymous; both terms are 


used in the literature. The idea of correlative coding was originated by Lender (1963). 
Lender’s work was generalized for binary data transmission by Kretzmer (1966). For fur- 
ther details on correlative coding techniques, see the book by Gitlin, Hayes, and Weinstein 
(1992); see also the papers by Pasupathy (1977), Kabal and Pasupathy (1975), and Sousa 
and Pasupathy (1983). 


. The material on digital subscriber lines presented in Section 4.8 is based on the two papers 


by Lin and Tzeng (1988), and Lechleider (1989), and the books by Starr, Cioffi, and 
Silverman (1999) and Chen (1998). 


. For a discussion of line codes for digital subscriber loops, see Gitlin et al. (1992). 


8. In Ericson (1971) it is shown that for every “reasonable” performance criterion, the opti- 


10. 


11. 


12. 


mum receiver can be realized as a matched filter followed by a tapped-delay-line equalizer, 
as shown in Figure 4.27. In addition to the mean-square error criterion considered in 
Section 4.9, reasonable performance criteria of interest include the following: 


(i) Minimization of the probability of symbol error. 
(ii) Zero-forcing equalization (to reduce the intersymbol interference to zero), followed 
by minimization of the probability of symbol error subject to this constraint. 
(iii) Minimization of signal-to-noise ratio at the sampling instants. 


Criterion (i) is the most natural approach to the optimization of a linear receiver; this 
approach, pursued in Aaron and Tufts (1966), is, unfortunately, complicated. Criterion 
(ii), due to Lucky et al. (1968), is a much simpler approach. Criterion (iii) is due to George 
(1965). 


. Adaptive equalization of telephone channels was pioneered by Lucky (1965, 1966). Since 


that time, numerous adaptive equalization schemes have been published in the literature, 
which provide equalization for specific synchronous data-transmission systems. For review 
papers on adaptive equalization, see Proakis (1975) and Qureshi (1982, 1985). Adaptive 
equalization is also discussed in detail in the books by Gitlin, Hayes, and Weinstein (1992, 
Chapter 8) and Proakis (1995, Chapter 6). 


It appears that early work on fractionally spaced equalizers was initiated by Brady (1970). 
Other contributions to the subject include subsequent work by Ungerboeck (1976) and 
Gitlin and Weinstein (1981). A detailed discussion of fractionally spaced equalizers is also 
presented in Gitlin et al. (1992). 


The LMS algorithm was originated by Widrow and Hoff, Jr. (1960). For a detailed con- 
vergence analysis of the LMS algorithm, see Haykin (1996, Chapter 9), and Widrow and 
Stearns (1985, Chapter 6). 


Decision-feedback equalization was first described in a report by Austin (1967). The op- 
timization of the decision-feedback equalizer for minimum mean-square error was first 
accomplished by Monsen (1971). A readable account of decision-feedback equalization is 
presented in the book by Gitlin, Hayes, and Weinstein (1992, pp. 500-510). 

Tomlinson (1971) and Harashima and Miyakawa (1972) describe a device for elim- 
inating error propagation in a decision-feedback equalizer. The device, known as the Tom- 
linson-Harashima precoder, appears in the transmitter as a preprocessor to the modulator. 
The basic idea of this precoder is to move the feedback section in the decision feedback 
equalizer to the transmitter where it is impossible to make decision errors. However, this 
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13. 


14. 


i PROBLEMS 


modification may result in a significant increase in transmit power; modulo arithmetic is 
used to overcome most of this power increase. 


For performance comparison between linear equalizers and decision-feedback equalizer, 
see Gitlin et al. (1992) and Proakis (1995). 


The intuitive discussion on error propagation in decision-feedback equalizers presented in 
Section 4.10 follows Gitlin et al. (1992). 

For a rigorous evaluation of the probability of symbol error P, ina decision-feedback 
equalizer with error propagation, see Duttweiler et al. (1974). In this paper it is showy 
that in the worst-case intersymbol interference, P, is multiplied by a factor of 2" relative 
to the probability of error that results in the absence of decision errors at high signal-to. 
noise ratios, where L is the number of taps in the feedback section. The result derived by 
Duttweiler et al. provides theoretical justification for the intuitive arguments presented in 
Section 4.10. 


Matched Filters 
4,1 Consider the signal s(t) shown in Figure P4.1. 


(a) Determine the impulse response of a filter matched to this signal and sketch it as a 
function of time. 


(b) Plot the matched filter output as a function of time. 
(c) What is the peak value of the output? 





Figure P4.1 


4.2 Figure P4.2a shows a pair of pulses that are orthogonal to each other over the interval 


[0, T]. In this problem we investigate the use of this pulse-pair to study a two-dimensional 
matched filter. 


(a) Determine the matched filters for the pulses s;(#) and s(t) considered individually; 
for’s,(£) the filter is the same as that considered in Problem 4.1. 
(b) Form a two-dimensional matched filter by connecting the two matched filters of Patt 
(a) in parallel, as shown in Figure P4.2b, Hence, demonstrate the following: 
(i) When the pulse s,(£) is applied to this two-dimensional filter, the response of the 
lower matched filter is zero. 
(ii) When the pulse s,(t) is applied to the two-dimensional filter, the response of the 
upper matched filter is zero. 
Generalize the results of your investigation. 
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4.3 Consider a rectangular pulse defined by 


(t) = A, O=tsT 
8 0, otherwise 


It is proposed to approximate the matched filter for g(t) by an ideal low-pass filter of 

bandwidth B; maximization of the peak pulse signal-to-noise ratio is the primary 

objective. 

(a) Determine the optimum value of B for which the ideal low-pass filter provides the 

best approximation to the matched filter. 

(b) By how many decibels is the ideal low-pass filter worse off than the matched filter? 
4.4 In this problem we explore another method for the approximate realization of a matched 

filter, this time using the simple resistance-capacitance (RC) low-pass filter shown in Fig- 

ure P4.4. The frequency resonse of this filter is 


7 1 
HO TS iio 


where fy = 1/27RC. The input signal g(t) is a rectangular pulse of amplitude A and 
duration T. The requirement is to optimize the selection of the 3-dB cutoff frequency fo 
of the filter so that the peak pulse signal-to-noise ratio at the filter output is maximized. 
With this objective in mind, show that the optimum value of fo is 0.2/T, for which the 
loss in signal-to-noise ratio compared to the matched filter is about 1 dB. 


R 
o wS] s oe 
Input c Output 
signal | signal 


FIGURE P4.4 





Probability of Error Calculation 


4.5 The formula for the optimum threshold in the receiver of Figure 4.4 is, in general, given 
by Equation (4.37). Discuss, in graphical terms, how this optimum choice affects the 
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4.6 


4.7 


4.8 


4.9 


4.10 


4.11 


contributions of the two terms in Equation (4.35) for the average probability of symbol 
error P, by considering the following two cases: 


(a) Po > Di 
(b) p1 < Po 


where po and p; are the a priori probabilities of symbols 0 and 1, respectively. 


In a binary PCM system, symbols 0 and 1 have a priori probabilities py and p,, respec. 
tively. The conditional probability density function of the random variable Y (with sample 
value y) obtained by sampling the matched filter output in the receiver of Figure 4.4 at the 
end of a signaling interval, given that symbol 0 was transmitted, is denoted by Fely|o) 
Similarly, fy(y|1) denotes the conditional probability density function of Y, given that 
symbol 1 was transmitted. Let A denote the threshold used in the receiver, so that if the 
sample value y exceeds A, the receiver decides in favor of symbol 1; otherwise, it decides 
in favor of symbol 0. Show that the optimum threshold A,,., for which the average prob. 
ability of error is a minimum, is given by the solution of 


Fer(Acpe| 1) _ Po 
FrlAcp:|0) Pi 


A binary PCM system using polar NRZ signaling operates just above the error threshold 
with an average probability of error equal to 107%. Suppose that the signaling rate is 
doubled. Find the new value of the average probability of error. You may use Table A6.6 
to evaluate the complementary error function. 
A continuous-time sigrial is sampled and then transmitted as a PCM signal. The random 
variable at the input of the decision device in the receiver has a variance of 0.01 volts’, 
(a) Assuming the use of polar NRZ signaling, determine the pulse amplitude that must 
be transmitted for the average error rate not to exceed 1 bit in 10° bits. 
(b) If the added presence of interference causes the error rate to increase to 1 bit in 10° 
bits, what is the variance of the interference? 
A binary PCM wave uses unipolar NRZ signaling to transmit symbols 1 and 0; symbol 
1 is represented by a rectangular pulse of amplitude A and duration T,. The channel noise 
is modeled as additive, white and Gaussian, with zero mean and power spectral density 
N,/2. Assuming that symbols 1 and 0 occur with equal probability, find an expression 
for the average probability of error at the receiver output, using a matched filter as de- 
scribed in Section 4.3. 
Repeat Problem 4.9 for the case of unipolar return-to-zero signaling, in which case symbol 
1 is represented by a pulse of amplitude A and duration T,/2 and symbol 0 is represented 
by transmitting no pulse. 

Hence show that this unipolar type of signaling requires twice the average power 
of unipolar nonreturn-to-zero (i.e., on-off) signaling for the same average probability of 
symbol error. 

In this problem, we revisit the PCM receiver of Figure 4.4, but this time we consider the 
use of bipolar nonreturn-to-zero signaling, in which case the transmitted signal s(t) is 
defined by 

Binary symbol 1: s(é) = +A for0 < ts T 

Binary symbol 0: s(t) = 0,0<t=T 
Determine the average probability of symbol error P, for this receiver assuming that the 
binary symbols 0 and 1 are equiprobable. 


Raised Cosine Spectrum 


4.12 


The nonreturn-to-zero pulse of Figure P4.12 may be viewed as a very crude form of a 
Nyquist pulse. Compare the spectral characteristics of these two pulses. 
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4.13 Determine the inverse Fourier transform of the frequency function P( f) defined in Equa- 
tion (4.60). . 

4.14 An analog signal is sampled, quantized, and encoded into a binary PCM wave. The spec- 
ifications of the PCM system include the following: 

Sampling rate = 8 kHz 
Number of representation levels = 64 
The PCM wave is transmitted over a baseband channel using discrete pulse-amplitude 
` modulation. Determine the minimum bandwidth required for transmitting the PCM wave 
if each pulse is allowed to take on the following number of amplitude levels: 2, 4, or 8. 

4.15 Consider a baseband binary PAM system that is designed to have a raised-cosine spectrum 
P(f). The resulting pulse p(t) is defined in Equation (4.62). How would this pulse be 
modified if the system was designed to have a linear phase response? 

4.16 A computer puts out binary data at the rate of 56 kb/s. The computer output is transmitted 
using a baseband binary PAM system that is designed to have a raised-cosine spectrum. 
Determine the transmission bandwidth required for each of the following rolloff factors: 
a = 0.25, 0.5, 0.75, 1.0. 

4.17 Repeat Problem 4.16, given that each set of three successive binary digits in the computer 
output are coded into one of eight possible amplitude levels, and the resulting signal is 
transmitted using an eight-level PAM system designed to have a raised-cosine spectrum. 

4.18 An analog signal is sampled, quantized, and encoded into a binary PCM wave. The num- 
ber of representation levels used is 128. A synchronizing pulse is added at the end of each 
code word representing a sample of the analog signal. The resulting PCM wave is trans- 
mitted over a channel of bandwidth 12 kHz using a quaternary PAM system with raised- 
cosine spectrum. The rolloff factor is unity. 

(a) Find the rate (b/s) at which information is transmitted through the channel. 
(b) Find the rate at which the analog signal is sampled. What is the maximum possible 
value for the highest frequency component of the analog signal? 

4.19 A binary PAM wave is to be transmitted over a baseband channel with an absolute max- 
imum bandwidth of 75 kHz. The bit duration is 10 ys. Find a raised-cosine spectrum 
that satisfies these requirements. 


Correlative-Level Coding 


4.20 The duobinary, ternary, and bipolar signaling techniques have one common feature: They 
all employ three amplitude levels. In what way does the duobinary technique differ from 
the other two? 

4.21 The binary data stream 001101001 is applied to the input of a duobinary system. 

(a) Construct the duobinary coder output and corresponding receiver output, without a 
precoder. 
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4.22 
4.23 


4.24 


4.25 


4.26 


(b) Suppose that owing to error during transmission, the level at the receiver input pr 
duced by the second digit is reduced to zero. Construct the new receiver output, T 

Repeat Problem 4.21, assuming the use of a precoder in the transmitter. 

The scheme shown in Figure P4.23 may be viewed as a differential encoder (consistin 

of the modulo-2 adder and the 1-unit delay element) connected in cascade with a special 

form of correlative coder (consisting of the 1-unit delay element and summer), A Single. 
delay element is shown in Figure P4.23 since it is common to both the differential encode, 
and the correlative coder. In this differential encoder, a transition is represented by symboj 

0 and no transition by symbol 1. 

(a) Find the frequency response and impulse response of the correlative coder part of the 
scheme shown in Figure P4.23. 

(b) Show that this scheme may be used to convert the on-off representation of a bi 
sequence (applied to the input) into the bipolar representation of the sequence at the 
output. You may illustrate this conversion by considering the sequence 010001101, 

For descriptions of on-off, bipolar, and differential encoding of binary sequences, see 

Section 3.7. 


Bipolar representation 
of binary sequence 


Binary sequence 
of 1s and Os 





FIGURE P4.23 


Consider a random binary wave x(t) in which the 1s and Os occur with equal probability, 

the symbols in adjacent time slots are statistically independent, and symbol 1 is repre- 

sented by A volts and symbol 0 by zero volts. This on-off binary wave is applied to the 

circuit of Figure P4.23. 

(a) Using the result of Problem 4.23, show that-the power spectral density of the bipolar 
wave y(t) appearing at the output of the circuit equals 


Sx(f) = T,A? sin*(mfT,) sinc’(fTs) 

(b) Plot the power spectral densities of the on-off and bipolar binary waves, and compart 
them. 

The binary data stream 011100101 is applied to the input of a modified duobinary system. 

(a) Construct the modified duobinary coder output and corresponding receiver output 
without a precoder. 

(b) Suppose that due to error during transmission, the level produced by the third digit 
is reduced to zero. Construct the new receiver output. 

Repeat Problem 4.25 assuming the use of a precoder in the transmitter. 


M-ary PAM Systems 


4.27 


Consider a baseband M-ary system using M discrete amplitude levels. The receiver model 
is as shown in Figure P4.27, the operation of which is governed by the following 
assumptions: 


4.28 
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(a) The signal component in the received wave is 
t 
m(t) = > an, sine( 4 = n) 


where 1/T is the signaling rate in bauds. 

(b) The amplitude levels are a, = +A/2, +3A/2,..., +(M — 1)A/2 if M is even, and 
a, = 0, +A,...,+(M— 1)A/2 if Mis odd. ` 

(c) The M levels are equiprobable, and the symbols transmitted in adjacent time slots are 
statistically independent. 

(d) The channel noise w(t) is white and Gaussian with zero mean and power spectral 
density N,/2. 

(e) The low-pass filter is ideal with bandwidth B = 1/2T. 

(f) The threshold levels used in the decision device are 0, +A,..., +(M — 2)A/2 if M 
is even, and +A/2, +3A/2,..., +(M — 2)A/2 if M is odd. 

The average probability of symbol error in this system is defined by 


1 A 
p,=(1-= -2 
j ( i) ere; A 
where ø is the standard deviation of the noise at the input of the decision device. Dem- 


onstrate the validity of this general formula by determining P, for the following three 
cases: M = 2, 3, 4. 


Law-pass Decisian- 
m n a 





Threshold 
FIGURE P4.27 


Suppose that in a baseband M-ary PAM system with M equally likely amplitude levels, 
as described in Problem 4.27, the average probability of symbol error P, is less than 107 
so as to make the occurrence of decoding errors negligible. Show that the minimum value 
of received signal-to-noise ratio in such a system is approximately given by 


(SNR) sin = 7.8(M? — 1) 


Digital Subscriber Lines 


4.29 


4.30 


The amplitude distribution of cross-talk in a digital subscriber line may be modeled as 
Gaussian. Justify the validity of such a model. Hint: Typically, a cable contains many 
twisted pairs. 
(a) Derive the formula for the power spectral density of a transmitted signal using the 
2B1Q line code. 
(b) Plot the power spectrum of the following line codes: 
œ Manchester code 
® Modified duobinary code 
> Bipolar return-to-zero code 
>» 2B1Q code 
Hence compare the relative merits of these line codes for their suitability in a digital 
subscriber loop. 
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4,31 In this problem we use the LMS algorithm to formulate an adaptive echo canceller f 
use in a digital subscriber line. The basic principle of adaptive echo cancellation i, A 
synthesize a replica of the echo and subtract it from the returned signal in an ad apti 
manner, as illustrated in Figure P4.31. The synthesized echo, denoted by #[7], is generated 
by passing the transmitted signal through an adaptive filter that ideally matches the trang. 
fer function of the echo path. The returned signal, consisting of the sum of actual ech 
r[n] and the received signal x|⁄], may be viewed as the desired response for the adaptive 
filtering process. 

Using the LMS algorithm, formulate the equations that define the operation of the 
adaptive echo canceller in Figure P4.31. 


Received 
signal, x[n] 
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Ficure P4.31 
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Equalization 


4.32 Figure P4.32 shows the cascade connection ofa linear channel and a synchronous tapped- 
delay-line equalizer. The impulse response of the channel is denoted by c(t), and that of 
the equalizer is denoted by h(#). The h(t) is defined by 


N 
h(t) = >, w, dt — kT) 
RST : 
where T is the spacing between adjacent taps of the equalizer, and the w, are its tap- 
weights (coefficients). The impulse response of the cascaded system of Figure P4.32 is 
denoted by p(t). The p(t) is sampled uniformly at the rate 1/T. To eliminate intersymbol 
interference, we require that the Nyquist criterion for distortionless transmission be sat- 
isfied, as shown by 


T) = 1, n=0 
i = 0, n#0 


(a) By imposing this condition, show that the (2N + 1) tap-weights of the resulting zero 
forcing equalizer satisfy the following set of (2N + 1) simultaneous equations: 


5 7 1, n=0 
fiat 
En VEE lo, on # £1, 42,...,4N 


where c, = c(nT). Hence, show that the zero-forcing equalizer is an inverse filter in 
that its transfer function is equal to the reciprocal of the transfer function of the 
channel. 

(b) A shortcoming of the zero-forcing equalizer is noise enhancement that can result ia 
poor performance in the presence of channel noise. To explore this phenomeno™ 
consider a low-pass channel with a notch at the Nyquist frequency, that is, Hf) 
zero at f = 1/2T. Assuming that the channel noise is additive and white, show a 
the power spectral density of the noise at the equalizer output app. roaches infinity at 
f = 1/2T. 


4.33 


4.34 


4.35 
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Even if the channel has no notch in its frequency response, the power spectral 
density of the noise at the equalizer output can assume high values. Justify the validity 
of this general statement. 


Received 
Transmitted Linear signa Tapped-delay-line 
signai channel equalizer Output 


Ficure P4.32 








Consider Equation (4.108), which defines the impulse response of a minimum mean- 

square error receiver. 

(a) Justify the validity of Equation (4.109) that is the Fourier-transformed version of 
Equation (4.108). 

(b) The power spectral density S,({f) in Equation (4.109) is the Fourier transform of the 
autocorrelation R,(7;, 72) of the time function q(t). The R,(71, 72) is defined by Equa- 
tion (4.98). Starting with Equation (4.98), derive the formula of Equation (4.111). 

Some radio systems suffer from multipath distortion, which is caused by the existence of 

more than one propagation path between the transmitter and the receiver. Consider a 

channel the output of which, in response to a signal s(t), is defined by (in the absence of 

noise) 


x(t) = ays(t —to1) + a2s(t — to) 
where a, and a2 are constant, and tọ, and toz represent transmission delays. It is proposed 
to use the three-tap delay-lie-filter of Figure P4.34 to equalize the multipath distortion 
produced by this channel. 
(a) Evaluate the transfer function of the channel. 


(b) Evaluate the parameters of the tapped-delay-line filter in terms of a4, a2, to, and toz, 
assuming that a) << a, and toz > toi. 





Input 
signal 

















FIGURE P4.34 


Let the sequence (x(#T)} denote the input applied to a tapped-delay-line equalizer. Show 
that intersymbol interference is eliminated completely by the equalizer provided that its 
frequency response satisfies the condition 


T 


Hf) = s 
(f) 3 X(f — HT) 
where T is the symbol duration. 
As the number of taps in the equalizer approaches infinity, the frequency response 
of the equalizer becomes a Fourier series with real coefficients and can therefore approx- 
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4.36 


4.37 


imate any function in the interval (-1/2T, 1/2T). Demonstrate this property of the 

equalizer. 

The step-size parameter p plays a critical role in the operation of the LMS algorithm, In 

this context, discuss the following two issues: 

(a) Stability. If u exceeds a certain critical value, the algorithm diverges (i-e., the system 
becomes unstable). 

(b) Memory. The reciprocal of may be viewed as a measure of the algorithm’s memory, 
As we make u smaller, more of the past samples of the input signal influence operation 
of the algorithm. 


Let the vectors w[] and w®?)[n] denote the tap-weights of the feed-forward and feed. 


back sections of the decision-feedback equalizer in Figure 4.32. Formulate the LMS al- 
gorithm for adjusting the tap-weights of this equalizer. 


Computer Experiments 


4.38 


4.39 


In Section 4.11 we studied the eye diagram of a quaternary (M = 4) PAM baseband 
transmission system under both noisy and band-limited conditions. In that experiment, 
the channel was assumed linear. In a strictly linear system with truly random data, all the 
eye openings would be identical. In practice, however, it is often possible to discern asym- 
metries in the eye pattern, which are caused by nonlinearities in the communication 
channel. 

In this experiment, we study the effect of a nonlinear channel on the openings of 
an eye pattern. Specifically, we repeat the computer experiment pertaining to the noiseless 
eye pattern of Figure 4.34a for M = 4, but this time assume that the channel is nonlinear 
with the following input-output relation: 


x(t) = s(t) + as?(t) 


where s(t) is the channel input and x(t) is the channel output, and a is a constant. 

(a) Do the experiment for a = 0, 0.05, 0.1, 0.2. 

(b) Hence, discuss how varying @ affects the shape of the eye pattern. 

In this experiment we study the root raised-cosine pulse due to Chennakeshu and Saulnier 

(1993). This pulse, denoted by p(t), has the following properties: 

> The pulse p(t) is symmetric in time, that is, p(—f) = p(t). 

> The squared Fourier transform of p(t), namely, P?(f), satisfies the raised cosine spectrum 
of Equation (4.60), but the Fourier transform P(f) itself does not. 

> The pulse p(t) satisfies the orthogonality constraint: 


ie plt)p(t — nT) dt = 0, n = £1, 22,245 


where T is the symbol period. 

(a) Compute the baseband waveform of the binary data stream 101100 for rolloff factor 
a = 0.3. 

(b) Compare the waveform computed in part (a) with that obtained using the ordinary 
raised-cosine spectrum. 





SIGNAL-SPACE ANALYSIS 


This chapter discusses some basic issues that pertain to the transmission of signals over an 
additive white Gaussian noise (AWGN) channel. Specifically, it addresses the following 


topics: 


p Geometric representation of signals with finite energy, which provides a mathematically 
elegant and highly insightful tool for the study of data transmission. 


p Maximum likelihood procedure for the detection of a signal in AWGN channel. 


p Derivation of the correlation receiver that is equivalent to the matched filter receiver 
discussed in the previous chapter. 


» Probability of symbol error and the union bound for its approximate calculation. 


The material presented herein naturally leads to the study of passband data transmission 
covered in Chapter 6. 


| 5.1 Introduction 


Consider the most basic form of a digital communication system depicted in Figure 5.1. 
A message source emits one symbol every T seconds, with the symbols belonging to an 
alphabet of M symbols denoted by m, 2, ... » my. Consider, for example, the remote 
connection of two digital computers, with one computer acting as an information source 
that calculates digital outputs based on observations and inputs fed into it. The resulting 
computer output is expressed as a sequence of Os and 1s, which are transmitted to a second 
computer over a communication channel. In this case, the alphabet consists simply of two 
binary symbols: 0 and 1. A second example is that of a quaternary PCM encoder with an 
alphabet consisting of four possible symbols: 00, 01, 10, and 11. In any event, the a priori 
probabilities p4, p2,..., pm specify the message source output. In the absence of prior 
information, it is customary to assume that the M symbols of the alphabet are equally 
likely. Then we may express the probability that symbol 7; is emitted by the source as 


pi = P(m,) 


1 (5.1) 
=p for t= L2... M 


The transmitter takes the message source output 7; and codes it into a distinct signal s,(t) 
suitable for transmission over the channel. The signal s;(t) occupies the full duration T 
allotted to symbol m;. Most important, s,(t) is a real-valued energy signal (i.e. a signal 
with finite energy), as shown by 


T 
E; = f s(t) dt, i=1,2,..., M (5.2) 
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Ficure 5.1 Block diagram of a generic digital communication system. 














The channel is assumed to have two characteristics: 


1. The channel is linear, with a bandwidth that is wide enough to accommodate the 
transmission of signal s;(t) with negligible or no distortion. 

2. The channel noise, w(t), is the sample function of a zero-mean white Gaussian noise 
process. The reasons for this second assumption are that it makes receiver calcula. 
tions tractable, and it is a reasonable description of the type of noise present in many 
practical communication systems. 


We refer to such a channel as an additive white Gaussian noise (AWGN) channel, Ac. 
cordingly, we may express the received signal x(t) as : 


OstsT 


x(t) = s(t) + wt), ti ET 


(5.3) 


and thus model the channel as in Figure 5.2. 

The receiver has the task of observing the received signal x(t) for a duration of T 
seconds and making a best estimate of the transmitted signal s;(t) or, equivalently, the 
symbol m;. However, owing to the presence of channel noise, this decision-making process 
is statistical in nature, with the result that the receiver will make occasional errors. The 
requirement is therefore to design the receiver so as to minimize the average probability 
of symbol error, defined as 


M 
P, = >, p: Pi + m, |m) (5.4) 
i=1 


where m; is the transmitted symbol, *# is the estimate produced by the receiver, and 
P(r # m; |m) is the conditional error probability given that the ith symbol was sent. The 
resulting receiver is said to be optimum in the minimum probability of error sense. 

This model provides a basis for the design of the optimum receiver, for which we 
will use geometric representation of the known set of transmitted signals, {s;(t)}. This 
method, discussed in Section 5.2, provides a great deal of insight, with considerable sim- 
plification of detail. 


Transmitted Received 
signal signal 
sft) + alt) 


White Gaussian noise 
Ww 


FIGURE 5.2 Additive white Gaussian noise (AWGN) model of a channel. 
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l 5.2 Geometric Representation of Signals 


The essence of geometric representation of signals! is to represent any set of M energy 
signals {s;(t)} as linear combinations of N orthonormal basis functions, where N = M. 
That is to say, given a set of real-valued energy signals s,(t), S2{t),..., Sae(t), each of 
duration T seconds, we write i 


= O0<t=T 
s;(t) = 2 shith { =1,2...,M (5.5) 
where the coefficients of the expansion are defined by 
2 i=1,2,...,M 
$y = | si(t)d,(t) dt, f 1,2... N (5.6) 


The real-valued basis functions $,(t), (t), ..., @x(t) are orthonormal, by which we 
mean 


1ifi=j 


7 
Oifiżj 2) 


iE, 

Í $;(t)4,(t) dt = Ô; = { 
where ô; is the Kronecker delta. The first condition of Equation (5.7) states that each basis 
function is normalized to have unit energy. The second condition states that the basis 
functions $4(f), $2(t),..., x(t) are orthogonal with respect to each other over the in- 
tervalO s t=T. 

The set of coefficients {s;}X; may naturally be viewed as an N-dimensional vector, 
denoted by s;. The important point to note here is that the vector s; bears a one-to-one 
relationship with the transmitted signal s;(t): 


e Given the N elements of the vectors s; (i.e., Sit» Sia) +++ Sin) Operating as input, we 
may use the scheme shown in Figure 5.3a to generate the signal s;(t), which follows 


sið 








pyi EO] 
(a) (b) 


Ficure 5.3 (a) Synthesizer for generating the signal s,(t). (b) Analyzer for generating the set of 
signal vectors {s;}. 
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directly from Equation (5.5). It consists of a bank of N multipliers, with each muj. 
tiplier having its own basis function, followed by a summer. This scheme may hę 
viewed as a synthesizer. 

> Conversely, given the signals s;(t), i = 1, 2,..., M, operating as input, we may use 
the scheme shown in Figure 5.3 to calculate the coefficients Sj, Si, «+ 3 Six which 
follows directly from Equation (5.6). This second scheme consists of a bank of y 
product-integrators or correlators with a common input, and with each one of them 
supplied with its own basis function. The scheme of Figure 5.3b may be viewed as 
an analyzer. 


Accordingly, we may state that each signal in the set {s;(t)} is completely determined 
by the vector of its coefficients 


s=|. |, i#=1,2,...,M (5.8) 


The vector s; is called a signal vector. Furthermore, if we conceptually extend our conven- 
tional notion of two- and three-dimensional Euclidean spaces to an N-dimensional Eu- 
clidean space, we may visualize the set of signal vectors {s;|i = 1, 2,..., M} as defining 
a corresponding set of M points in an N-dimensional Euclidean space, with N mutually 
perpendicular axes labeled 1, 62, ..., by. This N-dimensional Euclidean space is called 
the signal space. 

The idea of visualizing a set of energy signals geometrically, as just described, is of 
profound importance. It provides the mathematical basis for the geometric representation 
of energy signals, thereby paving the way for the noise analysis of digital communication 
systems in a conceptually satisfying manner. This form of representation is illustrated in 
Figure 5.4 for the case of a two-dimensional signal space with three signals, that is, N = 2 
and M = 3. 

In an N-dimensional Euclidean space, we may define lengths of vectors and angles 
between vectors. It is customary to denote the length (also called the absolute value or 
norm) of a signal vector s; by the symbol || s; ||. The squared-length of any signal vector s, 
is defined to be the inner product or dot product of s; with itself, as shown by 


Ils: l|? 


sis; 


It 


(9.9) 


N 

2 A 
Xs i=1,2,...,M 
mat 


where s; is the jth element of s; and the superscript T denotes matrix transposition. 

There is an interesting relationship between the enetgy content of a signal and its 
representation as a vector. By definition, the energy of a signal s;(t) of duration T seconds 
is 


E; =Í s?(t) dt (5.10) 


o 
Therefore, substituting Equation (5.5) into (5.10), we get 


TDN N 
E= I È saol sadio |a 


j=l k=1 
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-2 





-3 


FIGURE 5.4 Illustrating the geometric representation of signals for the case when 
N=2andM=3, i 


Interchanging the order of summation and integration, and then rearranging terms, we get 


N N T y% 
E; = 2 > SySik Í dilt), (t)dt (5.11) 


But since the ¢,(t) form an orthonormal set, in accordance with the two conditions of 
Equation (5.7), we find that Equation (5.11) reduces simply to 


N 

= 2 

E; = > Sij 
j=1 


Is: l)? 


(5.12) 


l 


Thus Equations (5.9) and (5.12) show that the energy of a signal s;(t) is equal to the 
squared length of the signal vector s;(t) representing it. 

In the case of a pair of signals s;(t) and s,(t), represented by the signal vectors s; and 
Sg, respectively, we may also show that 


Í s,(t)sg(t) dt = s7sk (5.13) 


Equation (5.13) states that the inner product of the signals s;(t) and s,(t) over the interval 
[0, T], using their time-domain representations, is equal to the inner product of their 
respective vector representations s; and s,. Note that the inner product of s;(t) and s; (t) is 
invariant to the choice of basis functions {¢,(t)}™, in that it only depends on the compo- 
nents of the signals s;(t) and s,(t) projected onto each of the basis functions. 
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Yet another useful relation involving the vector representations of the signals silt) 
: à 
and s; (t) is described by 


\| si — sal? = 2 (sy — Sei) 
ee (5.14) 
= i (s:t) — są(t)¥ dt 


0 


where ||s;— s || is the Euclidean distance, dig, between the points represented by the 
signal vectors s; and sg. 

To complete the geometric representation of energy signals, we need to have a rep. 
resentation for the angle 6, subtended between two signal vectors s; and s,. By definition, 
the cosine of the angle Oir is equal to the inner product of these two vectors divided by the 
product of their individual norms, as shown by 


sTsg 
cos bg = —— r 5. 
e= Tol isal (5415) 


The two vectors s; and s, are thus orthogonal or per, endicular to each other if their i 
. . . 8 . +e - . aoe Inner 
product s/s, is zero, in which case 64 = 90 degrees; this condition is intuitively satisfying, 


b EXaMPLe 5.1 Schwarz Inequality 


Consider any pair of energy signals s(t) and s2(¢). The Schwarz inequality states that 


es 2 æ 0 
( [. silted) = ( F stndt) ( ie sdt) (5.16) 


The equality holds if and only if s2(t) = cs4(t), where c is any constant. 
To prove this important inequality, let s(t) and s2(¢) be expressed in terms of the par 
of orthonormal basis functions ¢,(t) and ¢2(t) as follows: 
si(t) = siibi(t) + s12ġ2(t) 
S(t) = Saiba(t) + S22$2{2) 
where $4(t) and @,(t) satisfy the orthonormality conditions over the entire time interval 
(—%, &): 
E 1 forj =i 
iis Hilt) G(t)dt = ð; = { ie 


On this basis, we may represent the signals s(t) and s2(t) by the following respective pait of 
vectors, as illustrated in Figure 5.5: 


0 otherwise 


$21 
S2 = 
$22. 


From Figure 5.5 we readily see that angle @ subtended between the vectors s; and sz is 


cos g eee ee 
sill Ise ll 
1 


sı(t)s2(t}dt 


(5.17) 


0 1/2 on 2 
(f stndt) ( f siar) 
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FIGURE 5.5 Vector representations of signals s;(t) and s(t), providing the background picture 
for proving the Schwarz inequality. 


where we have made use of Equations (5.15), (5.13) and (5.9). Recognizing that | cos 6| = 1, 
the Schwarz inequality of Equation (5.16) immediately follows from Equation (5.17), More- 
over, from the first line of Equation (5.17) we note that |cos 8| = 1 if and only if s2 = cs, 
that is, s2(t) = cs,(t), where c is an arbitrary constant. 

The proof of the Schwarz inequality, as presented here, applies to real-valued signals. 
It may be readily extended to complex-valued signals, in which case Equation (5.16) is refor- 


mulated as 
2 æ o 
=(f seas) ss) a) (5.18) 


where the equality holds if and only if s2(t) = cs,(t), where c is a constant; see Problem 5.9. 
It is the complex form of the Schwarz inequality that was used in Chapter 4 to derive the 
matched filter. q 





iz $1(t)s3 (tat 





m GRAM-SCHMIDT ORTHOGONALIZATION PROCEDURE 


Having demonstrated the elegance of the geometric representation of energy signals, how 
do we justify it in mathematical terms? The answer lies in the Gram-Schmidt orthogon- 
alization procedure, for which we need a complete orthonormal set of basis functions. To 
proceed with the formulation of this procedure, suppose we have a set of M energy signals 
denoted by s,(t), so(t),..., Sa(t). Starting with s,(t) chosen from this set arbitrarily, the 
first basis function is defined by 

Si(t) 


Pil) = VE, (5.19) 
1 


where E, is the energy of the signal s,(t). Then, clearly, we have 


silt) = VEipilt) 





5.20 
= sudilt) ( ) 
where the coefficient s,; = VE, and ¢,(t) has unit energy, as required. 
Next, using the signal s(t), we define the coefficient sz; as 
E 
$21 = i So{t)p(t)dt f (5.21) 


We may thus introduce a new intermediate function 


Balt) = s2(t) — sz21$1(t) (5.22) 
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which is orthogonal to }1(é) over the interval 0 = t = T by virtue of Equation (5.21) an 
the fact that the basis function $;(t) has unit energy. Now, we are ready to define the 
second basis function as 


$2{t) = ol (5.23) 


Í glt)dt 


0 
Substituting Equation (5.22) into (5.23) and simplifying, we get the desired result 


2(t) — Sar Pilt, 
$-t) = se sat (5.24) 


where E, is the energy of the signal s,(z). It is clear from Equation (5.23) that 


T 
É pilt)ìdt = 1 


and from Equation (5.24) that 


T 
[, dseroatedt = 0 


That is to say, #1(t) and 2(t) form an orthonormal pair, as required. 
Continuing in this fashion, we may in general define 
i-1 
gilt) = silt) — È silt) (5.25) 


j=1 


where the coefficients s; are themselves defined by 
T 
Sj F i s;(t)ġ;(t)dt, j=1,2,...,i—1 (5.26) 


Equation (5.22) is a special case of Equation (5.25) with i = 2. Note also that for i = 1, 
the function g;(t) reduces to s;(t). 
Given the g;(t), we may now define the set of basis functions 


g(t) 


Q(t) = T > 
i, gilt)at 


which form an orthonormal set. The dimension N is less than or equal to the number of 
given signals, M, depending on one of two possibilities: 


i=1,2,...,N (5.27) 





> The signals s,(t), s2(t),---» Su(é) form a linearly independent set, in which case 
N=M. 

> The signals s,(t), So(t), « . . » Sm(t) are not linearly independent, in which case N<M 
and the intermediate function g;(t) is zero for i > N. 
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TABLE 5.1 Amplitude 
Levels of the 2B1Q Code 


Signal 
—— Gray 
Symbol Amplitude code 

s(t) =3 00 
S2(t) -1 01 
sa(t) +1 11 
Salt) +3 10 


Note that the conventional Fourier series expansion of a periodic signal is an example 


of a particular expansion of the type described herein. Also, the representation of a band- 
limited signal in terms of its samples taken at the Nyquist rate may be viewed as another 
sample of a particular expansion of this type. However, two important distinctions should 
be made: 


1. 


2. 


The form of the basis functions (t), 62(t),..., x(t) has not been specified. That 
is to say, unlike the Fourier series expansion of a periodic signal or the sampled 
representation of a band-limited signal, we have not restricted the Gram-Schmidt 
orthogonalization procedure to be in terms of sinusoidal functions or sinc functions 
of time. 

The expansion of the signal s;(t) in terms of a finite number of terms is not an 
approximation wherein only the first N terms are significant but rather an exact 
expression where N and only N terms are significant. 


b EXAMPLE 5.2 2B1Q Code 


The 2B1Q code was described in Chapter 4 as the North American line code for digital 
subscriber lines. It represents a quaternary PAM signal as shown in the Gray-encoded alphabet 
of Table 5.1. The four possible signals, s(t), s2(2), $3(t), and s4(z), are amplitude-scaled versions 
of a Nyquist pulse. Each signal represents a dibit. We wish to find the vector representation 
of the 2B1Q code. 

This example is simple enough for us to solve it by inspection. Let $,(t) denote the 
Nyquist pulse, normalized to have unit energy. The $,(t) so defined is the only basis function 
for the vector representation of the 2B1Q code. Accordingly, the signal-space representation 
of this code is as shown in Figure 5.6. It consists of four signal vectors s1, S2, 3, and s4, which 
are located on the ¢,-axis in a symmetric manner about the origin. In this example, we thus 
have M = 4 and N = 1. 

We may generalize the result depicted in Figure 5.6 for the 2B1Q code as follows. The 
signal-space diagram of an M-ary pulse-amplitude modulated signal, in general, is one- 
dimensional with M signal points uniformly positioned on the only axis of the diagram. 4 





| 2 >k—1 1} 2 >| 


FIGURE 5.6 Signal-space representation of the 2B1Q code. 
water 
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5.3 Conversion of the Continuous 
AWGN Channel into a Vector Channel 





Suppose that the input to the bank of N product integrators or correlators in Figure § 3, 
is not the transmitted signal s;(t) but rather the received signal x(t) defined in accordance 
with the idealized AWGN channel of Figure 5.2. That is to say, 


OstsT 


j=1,2,...,M (5.28) 


x(t) = s(t) + w(t), { 
where w(t) is a sample function of a white Gaussian noise process W(t) of zero mean and 
power spectral density No/2. Correspondingly, we find that the output of correlator j, say, 
is the sample value of a random variable X;, as shown by , 


li 


t 
X; Í x(t)d(t)dt 
0 


s (5.29) 
= sj + Wh j=1,2,..., N 


The first component, s,, is a deterministic quantity contributed by the transmitted signal 
s;(t); it is defined by 


T 
sy = f, sodita (5.30) 


The second component, w;, is the sample value of a random variable W; that arises because 
of the presence of the channel noise w(t); it is defined by 


T 
wj = Í w(t)(t)dt (5.31) 


Consider next a new random process X'(t) whose sample function x’ (t) is related to 
the received signal x(t) as follows: 
N 


x'(t) = x(t) — > x;ġ;(t) (5.32) 


j=l 
Substituting Equations (5.28) and (5.29) into (5.32), and then using the expansion of 
Equation (5.5), we get 


Mz 


x'(t) = st) + w(t) — 
N 
= w(t) — X, wolt) (5.33) 


j=1 
= w'(t) 


(sy + wi) glt) 


ki 


The sample function x’ (t) therefore depends solely on the channel noise w(t). On the basis 
of Equations (5.32) and (5.33), we may thus express the received signal as 
N 


x(t) = >, xlt) + x'(t) 


a (5.34) 
= > x;Q;(t) + w(t) 


Accordingly, we may view w(t) as a sort of remainder term that must be included on the 
right to preserve the equality in Equation (5.34). Itis informative to contrast the expansio? 
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of the received signal x(t) given in Equation (5.34) with the corresponding expansion of 
the transmitted signal s;(t) given in Equation (5.5). The latter expansion is entirely deter- 
ministic, whereas that of Equation (5.34).is random (stochastic), which is to be expected. 


STATISTICAL CHARACTERIZATION OF THE CORRELATOR OUTPUTS 


We now wish to develop a statistical characterization of the set of N correlator outputs. 
Let X(t) denote the random process, a sample function of which is represented by the 
received signal x(t). Correspondingly, let X; denote the random variable whose sample 
value is represented by the correlator output x; j = 1, 2, . . . , N. According to the AWGN 
model of Figure 5.2, the random process X(t) is a Gaussian process. It follows therefore 
that X; is a Gaussian random variable for all j (see Property 1 of a Gaussian process, 
Section 1.8). Hence, X; is characterized completely by its mean and variance, which are 
determined next. 

Let W, denote the random variable represented by the sample value w; produced by 
the jth correlator in response to the white Gaussian noise component w(t). The random 
variable W, has zero mean, because the noise process W(t) represented by w(t) in the 
AWGN model of Figure 5.2 has zero mean by definition. Consequently, the mean of X; 
depends only on s;, as shown by 


Lx, = EX] 
= Ejs; + W; 
ee (535) 
= sy 
To find the variance of X;, we note that 
o%, = var[X;] 
= El(X; - sl (5.36) 


= E[W7] 


where the last line follows from Equation (5.29) with x; and w; replaced by X; and W, 
respectively. According to Equation (5.31), the random variable W; is defined by 


T 
LA =| Wit) d(t)dt 
0 


We may therefore expand Equation (5.36) as follows: 


T T 
o%, = elf wogiod | Wiese | 


T pT (5.37) 
= alf, Í oj (t)d{u) W(t) Wed 
Interchanging the order of integration and expectation: 
T /T 
o% = | f, dd, Cnetw(ey weeded 
(5.38) 


T pT 
-Í Í $(t)b(u)Rw(t, u)didu 
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where Ryy(t, u) is the autocorrelation function of the noise process W(t). Since this Noise 
is stationary, Rw(t, u) depends only on the time difference t — u. Furthermore, since the 
noise W(t) is white with a constant power spectral density No/2, we may express Ry(t, x) 
as follows [see Equation (1.95)]: 


Ru(t, u) = = ate — n) (539 


Therefore, substituting Equation (5.39) into (5.38), and then using the sifting Property of 
the delta function S(t), we get 


T pT 
o%, = > f | ;(t)d;(u) 5(t — u)dtdu 


5.40 
No (7 (5.40) 
=f, ites 
Since the ¢;(t) have unit energy, by definition, we finally get the simple result 

o%, = No for all j (5.41) 


This important result shows that all the correlator outputs denoted by X; with j = 1, 
2,..., N, have a variance equal to the power spectral density No/2 of the noise process 
Wit). f 
Moreover, since the ¢;(t) form an orthogonal set, we find that the X; are mutually 
uncorrelated, as shown by 


cov[XjX4] = EX; — ux)(Xe — ux) 
= E[(X; -= S;)(Xk J Sik)] 
= E[W;W;] 


T T 
= a| f, mio f, ww) daude 


T pT 
=|, | Olt) be(u)Rw(t, u)dtdu (5.42 
T rT 
z F Í Í Oj (t)b,(u) ô(t — u)dtdu 
T 
= No f glerda(ene 
=0, j#k 


Since the X, are Gaussian random variables, Equation (5.42) implies that they are also 
statistically independent (see Property 4 of a Gaussian Process, Section 1.8). 
Define the vector of N random variables 
Xi 
X. 
x=) (5.43) 
Xy 
whose elements are independent Gaussian random variables with mean values equal to ŝy 
and variances equal to No/2. Since the elements of the vector X are statistically indepe™ 
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dent, we may express the conditional probability density function of the vector X, given 
that the signal s;(t) or correspondingly the symbol m; was transmitted, as the product of 
the conditional probability density functions of its individual elements as shown by 


N 
fxlx|m,) = II fxm) i=1,2,...,M (5.44) 


where the vector x and scalar x; are sample values of the random vector X and random 
variable X;, respectively. The vector x is called the observation vector; correspondingly, 
x; is called an observable element. Any channel that satisfies Equation (5.44) is called a 
memoryless channel. i 

Since each X; is a Gaussian random variable with mean s; and variance No/2, we 
have 





1 1 j= 1 ie 
Fx{xj|m) =. WaNo e| No (x; si} i= 1; A ety M (5.45) 


Therefore, substituting Equation (5.45) into (5.44) yields 
N 
fix |m) = (aN)? a| -4 > (x - s’), i=1,2,...,M (5.46) 
o j=1 


It is now clear that the elements of the random vector X completely characterize the 
summation term 2,X;¢;(t), whose sample value is represented by the first term in Equation 
(5.34). However, there remains the noise term w’ (t) in this equation, which depends only 
on the channel noise w(t). Since the noise process W(t) represented by w(t) is Gaussian 
with zero mean, it follows that the noise process W’(t) represented by the sample function 
w'(t) is also a zero-mean Gaussian process. Finally, we note that any random variable 
W'(t,), say, derived from the noise process W’(t) by sampling it at time tą, is in fact 
statistically independent of the set of random variables {X;}; that is to say (see Problem 
5.10), 


(= 1,2,...,N 
4 (5.47) 


E[X;W'(t,)] = 0, l 
Since any random variable based on the remainder noise process W'(t) is independent of 
the set of random variables {X;} as well as the set of transmitted signals {s;(ż)}, Equation 
(5.47) states that the random variable W’(t,) is irrelevant to the decision as to which 
particular signal was actually transmitted. In other words, the correlator outputs deter- 
mined by the received signal x(t) are the only data that are useful for the decision-making 
process and, hence, represent sufficient statistics for the problem at hand. By definition, 
sufficient statistics summarize the whole of the relevant information supplied by an ob- 
servation vector. 
We may now summarize the results presented in this section by formulating the 
theorem of irrelevance: 


Insofar as signal detection in additive white Gaussian noise is concerned, only the 
projections of the noise onto the basis functions of the signal set {s;(¢)}M, affects 
the sufficient statistics of the detection problem; the remainder of the noise is 
irrelevant. ` 


As a corollary to this theorem, we may state that the AWGN channel of Figure 5.2 is 
equivalent to an N-dimensional vector channel described by the observation vector 


x=s tw, i=1,2,...,M (5.48) 
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where the dimension N is the number of basis functions involved in formulating the signa] 
vector s;. The individual components of the signal vector s; and noise vector w are defined 
by Equations (5.6) and (5.31), respectively. The theorem of irrelevance and its corollary 
are indeed basic to the understanding of the signal detection problem as described next, 


a5 4 Likelihood Functions 








The conditional probability density functions fx(x|m;), i = 1, 2,..., M, are the very 
characterization of an AWGN channel. Their derivation leads to a functional dependence 
on the observation vector x, given the transmitted message symbol m,. However, at the 
receiver we have the exact opposite situation: We are given the observation vector x and 
the requirement is to estimate the message symbol m, that is responsible for generating x, 
To emphasize this latter viewpoint, we introduce the idea of a likelihood function, denoted 


by L(m,) and defined by 
L(m;) = f(x |), i= 1, 2, eees M (5.49) 


It is important however to recognize that although the L(m,) and fx(x|m,) have exactly 
the same mathematical form, their individual meanings are different. 

In practice, we find it more convenient to work with the log-likelibood function, 
denoted by /(m;) and defined by 


Km) = log Lim), i= 1,2,...,M (5.50) 


The log-likelihood function bears a one-to-one relationship to the likelihood function for 
two reasons: 


1. By definition, a probability density function is always nonnegative. It follows there- 
fore that the likelihood function is likewise a nonnegative quantity. 
2. The logarithmic function is a monotonically increasing function of its argument. 


The use of Equation (5.46) in (5.50) yields the log-likelihood functions foran AWGN 
channel as 


N 
lm) = z 3S (x, - 5), i=1,2,...,M (5.51) 
No jai 
where we have ignored the constant term —(N/2) log(7No) as it bears no relation what- 
soever to the message symbol m;. Note that the s,,7 = 1,2,..., N, are the elements of 
the signal vector s; representing the message symbol m;. With Equation (5.51) at our 
disposal, we are now ready to address the basic receiver design problem. 


5.5 Coherent Detection of Signals in Noise: 
Maximum Likelihood Decoding 




















Suppose that in each time slot of duration T seconds, one of the M possible signals s1(#), 
s(t), . .., Sy(t) is transmitted with equal probability, 1/M. For geometric signal represen- 
tation, the signal s;(t), ¿ = 1,2,..., M, is applied to a bank of correlators, with a common 
input and supplied with an appropriate set of N orthonormal basis functions. The resulting 
correlator outputs define the signal vector s;. Since knowledge of the signal vector s; is 4 
good as knowing the transmitted signal s,(t) itself, and vice versa, we may represent slt) 
by a point in a Euclidean space of dimension N = M. We refer to this point as the érars 
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mitted signal point or message point. The set of message points corresponding to the set 
of transmitted signals {s,(¢))4, is called a signal constellation. 

However, the representation of the received signal x(t) is complicated by the presence 
of additive noise w(t). We note that when the received signal x(t) is applied to the bank 
of N correlators, the correlator outputs define the observation vector x: From Equation 
(5.48), the vector x differs from the signal vector s; by the noise vector w whose orientation 
is completely random. The noise vector w is completely characterized by the noise w(t); 
the converse of this statement, however, is not true. The noise vector w represents that 
portion of the noise w(ż) that will interfere with the detection process; the remaining 
portion of this noise, denoted by w’(t), is tuned out by the bank of correlators. 

Now, based on the observation vector x, we may represent the received signal x(t) 
by a point in the same Euclidean space used to represent the transmitted signal, We refer 
to this second point as the received signal point. The received signal point wanders about 
the message point in a completely random fashion, in the sense that it may lie anywhere 
inside a Gaussian-distributed “cloud” centered on the message point. This is illustrated in 
Figure 5.7a for the case of a three-dimensional signal space. For a particular realization 
of the noise vector w (i.e., a particular point inside the random cloud of Figure 5.7a), the 
relationship between the observation vector x and the signal vector s; is as illustrated in 
Figure 5.76. 

We are now ready to state the signal detection problem: 


Given the observation vector x, perform a mapping from x to an estimate 1 of the 
transmitted symbol, m;, in a way that would minimize the probability of error in 
the decision-making process. 


Suppose that, given the observation vector x, we make the decision 7% = m;. The 

probability of error in this decision, which we denote by P.(,| x), is simply 
P.(m,| x) = P(m; not sent |x) 

5.52 

= 1 — P(m; sent|x) ( 


The decision-making criterion is to minimize the probability of error in mapping each 
given observation vector x into a decision. On the basis of Equation (5.52), we may there- 
fore state the optimum decision rule: 


Set #% = m; if 














5.53 
P(m, sent|x) = P(m, sent|x) for all k # i (333) 
$2 $z : 
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FIGURE 5.7 Illustrating the effect of noise perturbation, depicted in (a), on the location of the 
received signal point, depicted in (b). 
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where k = 1, 2,..., M. This decision rule is referred to as the maximum a Posteriori 
probability (MAP) rule. 

The condition of Equation (5.53) may be expressed more explicitly in terms of the 
a priori probabilities of the transmitted signals and in terms of the likelihood functions 
Using Bayes’ rule in Equation (5.53), and for the moment ignoring possible ties in the 
decision-making process, we may restate the MAP rule as follows: 


Set # = m; if 


paf xx |m) 


LAO is maximum for k = i 
fx(x) 
where p, is the a priori probability of transmitting symbol mg, fx(x |74) is the conditional 
probability density function of the random observation vector X given the transmission 
of symbol ,, and f(x) is the unconditional probability density function of X. In Equation 
(5.54) we may note the following: 


(5.54) 


> The denominator term f(x) is independent of the transmitted symbol. 

e The a priori probability p, = p; when all the source symbols are transmitted with 
equal probability. 

> The conditional probability density function fx(x|#,) bears a one-to-one relation. 
ship to the log-likelihood function (m). 


Accordingly, we may restate the decision rule of Equation (5.54) in terms of (m) simply 
as follows: : 

Set #2 = m; if 5.55 
lim} is maximum for k = i 5] 
This decision rule is referred to as the maximum likelihood rule, and the device for its 
implementation is correspondingly referred to as the maximum likelihood decoder. Ac- 
cording to Equation (5.55), a maximum likelihood decoder computes the log-likelihood 
functions as metrics for all the M possible message symbols, compares them, and then 
decides in favor of the maximum. Thus the maximum likelihood decoder differs from the 
maximum 4 posteriori decoder in that it assumes equally likely message symbols. 

It is useful to have a graphical interpretation of the maximum likelihood decision 
rule. Let Z denote the N-dimensional space of all possible observation vectors x. We refer 
to this space as the observation space. Because we have assumed that the decision rule 
must say #1 = m,, where i = 1,2,..., M, the total observation space Z is correspondingly 
partitioned into M-decision regions, denoted by Z,, Z2,..., Zm. Accordingly, we may 
restate the decision rule of Equation (5.55) as follows: 

Observation vector x lies in region Z; if (5.56) 


lony) is maximum for k = i 


Aside from the boundaries between the decision regions Z1, Z2,..., Zm, it is clear that 
this set of regions covers the entire space of possible observation vectors x. We adopt the 
convention that all ties are resolved at random; that is, the receiver simply makes a guess. 
Specifically, if the observation vector x falls on the boundary between any two decision 
regions, Z; and Z,, say, the choice between the two possible decisions 7 = m; and #1 = mM 
is resolved a priori by the flip of a fair coin. Clearly, the outcome of such an event does 
not affect the ultimate value of the probability of error since, on this boundary, the com 
dition of Equation (5.53) is satisfied with the equality sign. 
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The maximum likelihood decision rule of Equation (5.55) or its geometric counter- 
part described in Equation (5.56) is of a generic kind, with the channel noise w(t) being 
additive as the only restriction imposed on it. We next specialize this rule for the case when 
w(t) is both white and Gaussian. 

From the log-likelihood function defined in Equation (5.51) for an AWGN channel 
we note that /(#2,) attains its maximum value when the summation term 

N 

È (x; sy? 

j=l 
is minimized by the choice k = i. Accordingly, we may formulate the maximum likelihood 
decision rule for an AWGN channel as - 


Observation vector x lies in region Z; if 
(5.57) 
Se — sx) is minimum for k = i 


Next, we note from our earlier discussion that (see Equation (5.14) for comparison) 


N 
> (x; — Sy)? = || x — sa ||? (5.58) 
ja 


where ||x — są || is the Euclidean distance between the received signal point and message 
point, represented by the vectors x and s,, respectively. Accordingly, we may restate the 
decision rule of Equation (5.57) as follows: 


Observation vector x lies in region Z; if 


59 
the Euclidean distance || x ~ s, || is minimum for k = i (3:32) 


Equation (5.59) states that the maximum likelihood decision rule is simply to choose the 
message point closest to the received signal point, which is intuitively satisfying. 

In practice, the need for squarers in the decision rule of Equation (5.59) is avoided 
by recognizing that 


Me 


I 
may 


= sy)? -5 x? -2 > XjSej + 2 si (5.60) 

I 
The first summation term of this expansion is independent of the index k and may therefore 
be ignored. The second summation term is the inner product of the observation vector x 
and signal vector s,. The third summation term is the energy of the transmitted signal 
s(t). Accordingly, we may formulate a decision rule equivalent to that of Equation (5.59) 
as follows: 


ra vector x lies in region Z; if 


(5.61) 
> XjSkj — Fi is maximum for k = i 
where E, is the energy of the transmitted signal s, (t): 
N 
E, = >) si i (5.62) 
įj=1 


From Equation (5.61) we deduce that, for an AWGN channel, the decision regions 
ate regions of the N-dimensional observation space Z, bounded by linear [(N — 1)- 
dimensional hyperplane] boundaries. Figure 5.8 shows the example of decision regions for 
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FIGURE 5.8 Illustrating the partitioning of the observation space into decision regions for the 
case when N = 2 and M = 4; it is assumed that the M transmitted symbols are equally likely, 


M = 4 signals and N = 2 dimensions, assuming that the signals are transmitted with equal 
energy, E, and equal probability. 


i 5.6 Correlation Receiver 


From the material presented in the previous sections, we find that for an AWGN channel 
and for the case when the transmitted signals s;(t), s2,..., Sm(t) are equally likely, the 
optimum receiver consists of two subsystems, which are detailed in Figure 5.9 and de- 
scribed here: 


1. The detector part of the receiver is shown in Figure 5.9a. It consists of a bank of M 
product-integrators or correlators, supplied with a corresponding set of coherent 
reference signals or orthonormal basis functions $,(t), b2(t),..., @n(t) thar are 
generated locally. This bank of correlators operates on the received signal x(t), 
0 <tT, to produce the observation vector x. 

The second part of the receiver, namely, the signal transmission decoder is shown in 
Figure 5.95. It is implemented in the form of a maximum-likelihood decoder that 
operates on the observation vector x to produce an estimate, #, of the transmitted 
symbol m; i = 1, 2,..., M, in a way that would minimize the average probability 
of symbol error. In accordance with Equation (5.61), the N elements of the obser- 
vation vector x are first multiplied by the corresponding N elements of each of the 
M signal vectors 51, S2, . - . , Sm, and the resulting products are successively summed 
in accumulators to form the corresponding set of inner products {x"s4 |k = 1,2,-++ 
M}. Next, the inner products are corrected for the fact that the transmitted sig 
energies may be unequal. Finally, the largest in the resulting set of numbers is selected; 
and an appropriate decision on the transmitted message is made. 


bd 


The optimum receiver of Figure 5.9 is commonly referred to as a correlation receiver. 
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FIGURE 5.9 (a) Detector or demodulator. (b) Signal transmission decoder. 


= EQUIVALENCE OF CORRELATION AND MATCHED FILTER RECEIVERS 


The detector shown in Figure 5.94 involves a set of correlators. Alternatively, we may use 
a corresponding set of matched filters to build the detector; the matched filter and its 
properties were considered in Section 4.2. To demonstrate the equivalence of a correlator 
and a matched filter, consider a linear time-invariant filter with impulse response h,(t). 
With the received signal x(f) used as the filter input, the resulting filter output, y,(t), is 
defined by the convolution integral: 


y(t) = ‘a x()hj(t — t)dr (5.63) 
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FIGURE 5.10 Detector part of matched filter receiver; the signal transmission decoder is as 
shown in Fig. 5.9b. 


From the definition of a matched filter presented in Section 4.2, we recall that the impulse 
response ;(t) of a linear time-invariant filter matched to an input signal ¢,(t) is a time. 
reversed and delayed version of the input ¢,(t). Suppose that we set 


h(t) = ${T — t) (5.64) 
Then the resulting filter output is 


y(t) = Í x(r)@(T — t+ q)dr (5.65) 


=y 


Sampling this output at time è = T, we get 


y(T) = | x(nidy(ride 
Since, by definition, ¢,(t) is zero outside the interval 0 = ¢ = T, we find that y;(T) is in 
actual fact the jth correlator output x; produced by the received signal x(t) in Figure 5.94, 
as shown by 


Pig 
y(T) = i x(7);(r)dr (5.66) 


Accordingly, the detector part of the optimum receiver may also be implemented using & 
bank of matched filters, as shown in Figure 5.10. It is important to note, however, that 
the output of each correlator in Figure 5.9a is equivalent to the output of a corresponding 
matched filter in Figure 5.10 orily when that output is sampled at time ¢ = T. 


| 5.7 Probability of Error 


To complete the statistical characterization of the correlation receiver depicted in Fig? 
5.9, we need to evaluate its noise performance. To do so, suppose that the observation 
space Z is partitioned, in accordance with the maximum likelihood decision rule, into 4 
set if M regions (Z,}{,. Suppose also that symbol 7; (or, equivalently, signal vector 58 








5.7 Probability of Error 329 


transmitted, and an observation vector x is received. Then an error occurs whenever the 
received signal point represented by x does not fall inside region Z; associated with the 
message point represented by s;. Averaging over all possible transmitted symbols, we 
readily see that the average probability of symbol error, P, is 


P, = >, pi Pix does not lie in Z;| m; sent) 


1 M a 
S > P(x does not lie in Z;|7; sent) (5.67) 


1% 
= 1 ~ — J P(x lies in Z;|m; sent) 
Mm 
where we have used standard notation to denote the probability of an event and the 
conditional probability of an event. Since x is the sample value of random vector X, we 
may rewrite Equation (5.67) in terms-of the likelihood function (when m; is sent) as 
follows: 


1 iM 
oe Geary > [, fx(x|m) dx (5.68) 


For an N-dimensional observation vector, the integral in Equation (5.68) is likewise 
N-dimensional. 


w INVARIANCE OF THE PROBABILITY OF ERROR TO 
ROTATION AND TRANSLATION 


The way in which the observation space Z is partitioned into the set of regions Z4, Zz, +». s 
Zm, in the maximum likelihood detection of a signal in additive white Gaussian noise, is 
uniquely defined by the signal constellation under study. Accordingly, changes in the ori- 
entation of the signal constellation with respect to both the coordinate axes and origin of 
the signal space do not affect the probability of symbol error P, defined in Equation (5.68). 
This result is a consequence of two facts: 
1. In maximum likelihood detection, the probability of symbol error P, depends solely 
on the relative Euclidean distances between the message points in the constellation. 
2. The additive white Gaussian noise is spherically symmetric in all directions in the 
signal space. 

Consider first the invariance of P, with respect to rotation. The effect of a rotation 
applied to all the message points in a constellation is equivalent to multiplying the 
N-dimensional signal vector s; by an N-by-N orthonormal matrix denoted by Q for all i. 
The matrix Q satisfies the condition 


QQ? =I (5.69) 


where I is the identity matrix whose diagonal elements are all unity and its off-diagonal 
elements are all zero. Note that according to Equation (5.69), the inverse of a real-valued 
orthonormal matrix is equal to its transposed form. Thus the signal vector s; is replaced 
by its rotated version 


S rotate = Qs;, i= 1, 2, a | M (5.70) 
Correspondingly, the N-by-1 noise vector w is replaced by its rotated version 
Wrorate = QW (5.71) 
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However, the statistical characteristics of the noise vector are unaffected by this rotatj, 
for the following reasons: a 


> From Chapter 1 we recall that a linear combination of Gaussian random variables 
is also Gaussian. Since the noise vector w is Gaussian, by assumption, it follows that 
the rotated noise vector W,orate is also Gaussian. 


» Since the noise vector w has zero mean, the rotated-noise vector Wrorate also has zeto 
mean, as shown by 


E[Wroratel = E[Qw] 
= QE[w] (5.72) 
=0 
> The covariance matrix of the noise vector w is equal to (No/2)I, where No/2 is the 
power spectral density of the AWGN w(t); that is, 


nN 
Eww] =I (5.73) 


Hence, the covariance matrix of the rotated noise vector Wyotate 1S 
E[WrocateW rotate = E[Qw(Qw)"] 
= E[Qww7Q"| 
= QE|ww7]Q" (5.74) 


where in the last two lines we have made use of Equations (5.73) and (5.69). 


In light of these observations, we may express the observation vector for the rotated 
signal constellation as 
Xrotare = Qs; + w, i=1,2,...,M (5.75} 


From Equation (5.59) we know that the decision rule for maximum likelihood detection 
is based on the Euclidean distance from the observation vector X,ctare to the rotated signal 
VectOL Sirotare = Qs;- Comparing Equation (5.75) to Equation (5.48), we readily see that 


Il Xrotate — Så rotate I = I X~ |i for all i (5.76) 


We may therefore formally state the principle of rotational invariance as follows: 


If a signal constellation is rotated by an orthonormal transformation, that is, 
Sirotare = Qs; i= 1, 2, E i M 


where Q is an orthonormal matrix, then the probability of symbol error P, incurred 
in maximum likelihood signal detection over an AWGN channel is completely 
unchanged. 


We illustrate this principle with an example. The signal constellation shown in Figure 
5.11b is the same as that of Figure 5.11a, except that it has been rotated through 4 
degrees, Although these two constellations do indeed look different, the principle of ro 
tational invariance tells us immediately that the P, is the same for both of them. 
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FIGURE 5.11 A pair of signal constellations for illustrating the principle of rotational invariance. 


Consider next the issue of invariance to translation. Suppose all the message points 
in a signal constellation are translated by a constant vector amount a, as shown by 
Si translate — S; — a, i= 1, 2, .. M (5.77) 
The observation vector is correspondingly translated by the same vector amount, as shown 
by 
Xranslate — X — a (5.78) 
From Equations (5.77) and (5.78) we see that the translate a is common to both the 


translated signal vector s; and translated observation vector x. We therefore immediately 
deduce that 


I Xtranslate T Si translate | = I X— S; I for all i (5.79) 


and thus formulate the principle of translational invariance as follows: 


If a signal constellation is translated bya constant vector amount, then the prob- 
ability of symbol error P, incurred in maximum likelihood signal detection over an 
AWGN channel is completely unchanged. 


As an example, consider the two signal constellations shown in Figure 5.12, which 
pertain to a pair of different 4-level PAM signals. The constellation of Figure 5.126 is the 
same as that of Figure 5.12a, except for a translation of 3a/2 to the right along the 
¢,-axis. The principle of translational invariance says that the P, is the same for both of 
these constellations. 


m MINIMUM ENERGY SIGNALS 


A useful application of the principle of translational invariance is in the translation of a 
given signal constellation in such a way that the average energy is minimized. To explore 


ms e | s . $: s hd + Qi 
~3al2 -a/2 0 al2 3a/2 o a 2a 3a 











(a) (b) 


FIGURE 5,12 A pair of signal constellations for illustrating the principle of translational 
invariance. 
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this issue, consider a set of symbols m, m2, . . . , mm represented by the signal vectors, 
S2,- - +5 Sm, respectively. The average energy of this signal constellation translated by 
vector amount a is 


M 
Erransiate ~ > Il 8; — a Il ?p; (5.80) 


where p; is the probability that symbol 7n; is emitted by the source of information. The 
squared Euclidean distance between s; and a is expanded as 
ls =al? = ls? 2a7s, + |al? 


We may therefore rewrite Equation (5.80) in the expanded form 


M M M 
Beranslare = > Is: | 72: - 2 2 asp: + lall? 2 D; 
i=1 i=1 i=] (5.81) 
= € — 2aEjs] + | all? 
where € is the average energy of the original signal constellation, and 
M 
Els] = > sp; (5.82) 


i=1 


Differentiating Equation (5.81) with respect to the vector a and then setting the result 
equal to zero, we readily find that the minimizing translate is 


amin = E[s] (5.83) 
The minimum average energy of the signal constellation translated in this way is 
SE translate,min =E- || amin I = (5.84) 


We may now state the procedure for finding the minimum energy translate: 


Given a signal constellation {s,}“,, the corresponding signal constellation with min- 
imum average energy is obtained by subtracting from each signal vector s; in the 
given constellation an amount equal to the constant vector EJs], where E[s] is de- 
fined by Equation (5.82). 


Recalling that the energy (or power) needed for signal transmission is a primary resource, 
the minimum energy translate provides a principled method for translating a signal con- 
stellation of interest so as to minimize the energy requirement. For example, the constel- 
lation of Figure 5.12a has minimum average energy, whereas that of Figure 5.125 does 
not. 


UNION BOUND ON THE PROBABILITY OF ERROR? 


For AWGN channels, the formulation of the average probability of symbol error, Pes is 
conceptually straightforward. We simply write P, in integral form by substituting Equation 
(5.46) into Equation (5.68). Unfortunately, however, numerical computation of the inte- 
gral is impractical, except in a few simple (but important) cases. To overcome this com- 
putational difficulty, we may resort to the use of bounds, which are usually adequate to 
predict the signal-to-noise ratio (within a decibel or so) required to maintain a prescribe 

error rate. The approximation to the integral defining P, is made by simplifying the integral 
or simplifying the region of integration. In the sequel, we use the latter procedure to 4- 
velop a simple yet useful upper bound called the union bound as an approximation to the 
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average probability of symbol error for a set of M equally likely signals (symbols) in an 
AWGN channel. 

Let A;r, with (i, k) = 1, 2,..., M, denote the event that the observation vector x is 
closer to the signal vector s, than to s; when the symbol 7; (vector s,) is sent. The con- 
ditional probability of symbol.error when symbol 7; is sent, P.(7;), is equal to the prob- 
ability of the union of events, An, Ap,.--, Aij—1, Aijv1s ++, Azm. From probability 
theory we know that the probability of a finite union of events is overbounded by the sum 
of the probabilities of the constituent events. We may therefore write 


M 

Pm) = >) P(Aw), i=1,2,...,M (5.85) 
k=1 
Rei, 


This relationship is illustrated in Figure 5.13 for the case of M = 4. In Figure 5.1324, we 
show the four message points and associated decision regions, with the point sı assumed 
to represent the transmitted symbol. In Figure 5.13, we show the three constituent signal- 
space descriptions where, in each case, the transmitted message point s, and one other 
message point are retained. According to Figure 5.13a the conditional probability of sym- 
bol error, P,(m;), is equal to the probability that the observation vector x lies in the shaded 
region of the two-dimensional signal-space diagram. Clearly, this probability is less than 
the sum of the probabilities of the three individual events that x lies in the shaded regions 
of the three constituent signal spaces depicted in Figure 5.130. 

It is important to note that, in general, the probability P(A,,) is different from the 
probability P(# = m,|m;). The latter is the probability that the observation vector x is 

















FIGURE 5.13 Illustrating the union bound. (a) Constellation of four message points. (b) Three 
constellations with a common message point and one other message point retained from the origi- 
nal constellation. 
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closer to the signal vector s, than every other, when s; (or m;) is sent. On the other hand 
the probability P(Aj,) depends on only two signal vectors, s; and są. To emphasize this 
difference, we rewrite Equation (5.85) by adopting P2(s,, Są} in place of P(Ax). We thus 
write 


M 
Pm) = È Palsa s i= 1,2,...,M (5.86 


k#i 


The probability P2(s,, Są) is called the pairwise error probability in that if a data transmis. 
sion system uses only a pair of signals, s; and s,, then P(S; s) is the probability of the 
receiver mistaking s, for s; 

Consider then a simplified digital communication system that involves the use of two 
equally likely messages represented by the vectors s; and s,. Since white Gaussian noise js 
identically distributed along any set of orthogonal axes, we may temporarily choose the 
first axis in such a set as one that passes through the points s; and sg; for three examples, 
see Figure 5.13b. The corresponding decision boundary is represented by the bisector that 
is perpendicular to the line joining the points s; and są. Accordingly, when the symbol m 
(vector s;) is sent, and if the observation vector x lies on the side of the bisector where s 
lies, an error is made. The probability of this event is given by 


P2(s;, s4) = P(x is closer to s, than s; when s; is sent) 


æ 1 i (5.87) 
= p V co =) dv 
where d; is the Euclidean distance between s; and sz; that is, 
di, = || 5; — 5 || (5.88) 
From the definition of the complementary error function, we have 
2 

Vrda © 
Thus, in terms of this function, with z set equal to v/V No, we find that Equation (5.87) 
takes on the compact form 


erfc(u) = xp(~z") dz 





P,(s;, s4) = i ertz) (5.89) 
Substituting Equation (5.89) into Equation (5.86), we get 
Pim) = 1 5 ete dit ), i=1,2,...,M (5.90) 
er wae 


The probability of symbol error, averaged over all the M symbols, is therefore overbounded 
as follows: 


F (5.91) 





where p; is the probability of transmitting symbol 7; 
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There are two special forms of Equation (5.91) that we will find useful in Chapter 6 
on passband data transmission: 


1. Suppose that the signal constellation is circularly syrnmetric about the origin. Then 
the conditional probability of error P.(m,) is the same for all i, in which case Equation 
(5.91) reduces to 





M 
P< i 2 ere( 44) for alli (5.92) 
kei 


2. Define the minimum distance of a signal constellation, din as the smallest Euclidean 
distance between any two transmitted signal points in the constellation, as shown by 


Amin = Min dg for all i and k (5.93) 


ki 


Then, recognizing that the complementary error function erfc() is a monotonically de- 
creasing function of its argument u, we may write 








ere( 4) = < ete a for all i and k (5.94) 


We may therefore, in general, simplify the bound on the average probability of symbol 
error in Equation (5.91) as 








(M yii 1) dimin 
P, = 7 erfc ia (5.95) 
The complementary error function is itself bounded as? 
1 dhin 
erfc (4) = he exo( dhs) (5.96) 
Accordingly, we may further simplify the union bound on P, given in Equation (5.95) as 


p -M1 x (-2) 
e> ave OPN 4N 


Equation (5.97) shows that for a prescribed AWGN channel, the average probability of 
symbol error P, decreases exponentially as the squared minimum distance, dhin. 


(5.97) 


BIT VERSUS SYMBOL ERROR PROBABILITIES 


Thus far, the only figure of merit we have used to assess the noise performance of a digital 
passband transmission system has been the average probability of symbol error. This figure 
of merit is the natural choice when messages of length m = log,M are transmitted, such 
as alphanumeric symbols. However, when the requirement is to transmit binary data such 
as digital computer data, it is often more meaningful to use another figure of merit called 
the bit error rate (BER). Although, in general, there are no unique relationships between 
these two figures of merit, it is fortunate that such relationships can be derived for two 
cases of practical interest, as discussed next. 
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Case I 


In the first case, we assume that it is possible to perform the mapping from bina, 
to M-ary symbols in such a way that the two binary M-tuples corresponding to any Pair 
of adjacent symbols in the M-ary modulation scheme differ in only one bit position, This 
mapping constraint is satisfied by using a Gray code. When the probability of symbol 
error P, is acceptably small, we find that the probability of mistaking one symbol fo, 
either one of the two “nearest” symbols is much greater than any other kind of symbo| 
error. Moreover, given a symbol error, the most probable number of bit errors is op, 
subject to the aforementioned mapping constraint. Since there are log,M bits per symbol, 
it follows that the average probability of symbol error is related to the bit error rate a; 
follows: 





logıM 
P. = z( U {ith bit is in error) ) 
i=1 
log,M 
< P(ith bit is in error) (5.97) 
i=1 
= log, M - (BER) 
We also note that 
P, = P(ith bit is in error) = BER (5.98) 
It follows therefore that the bit error rate is bounded as follows: 
P._< BER <P. (5.99) 
log.M : ” 


Case 2 


Let M = 2¥, where K is an integer. We assume that all symbol errors are equally 
likely and occur with probability 


Pe _ OP 

M-1 2- 
where P, is the average probability of symbol error. What is the probability that the ith 
bit in a symbol is in error? Well, there are 257" cases of symbol error in which this 


particular bit is changed, and there are 2X~" cases in which it is not changed. Hence, the 
bit error rate is 





cal 00 
R= |= 5.1 
BE (3 — :)P. (5.100) 
or, equivalently, 
M/2 
= | — 5.101) 
BER G ~ :)P. ( 


Note that for large M, the bit error rate approaches the limiting value of P,/2. The same 
idea described here also shows that bit errors are not independent, since we have 
K-2 


P(ith and jth bits are in error) = ary 


= Pe # (BER)? 
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5.8 Summary and Discussion 


The primary goal of the material presented in this chapter is the formulation of a systematic 
procedure for the analysis and design of a digital communication receiver in the presence 
of additive white Gaussian noise (AWGN). The procedure, known as maximum likelihood 
detection, decides which particular transmitted symbol is the most likely cause of the noisy 
signal observed at the channel output. The approach that led to the formulation of the 
maximum likelihood detector (receiver) is called signal-space analysis. The basic idea of the 
approach is to represent each member of a set of transmitted signals by an N-dimensional 
vector, where N is the number of orthonormal basis functions needed for a unique geo- 
metric representation of the transmitted signals. The set of signal vectors so formed defines 
a signal constellation in an N-dimensional signal space. 

For a given signal constellation, the (average) probability of symbol error P, incurred 
in maximum likelihood signal detection over an AWGN channel is invariant to rotation 
of the signal constellation as well as its translation. However, except for a few simple (but 
important) cases, the numerical calculation of P, is an impractical proposition. To over- 
come this difficulty, the customary practice is to resort to the use of bounds that lend 
themselves to computation in a straightforward manner. In this context, we described the 
union bound that follows directly from the signal-space diagram. The union bound is based 
on an intuitively satisfying idea: The probability of symbol error P, is dominated by the 
nearest neighbors to the transmitted signal. The results obtained using the union bound 
are usually fairly accurate when the signal-to-noise ratio is high. 

With the material on signal-space analysis and related issues on hand, we are well- 
equipped to study passband data transmission systems, which we do in Chapter 6. 


i NOTES AND REFERENCES 


1. The geometric representation of signals was first developed by Kotel’nikov in 1947: V. A, 
Kotel’nikov, The Theory of Optimum Noise Immunity (Dover Publications, 1960), which 
is a translation of the original doctoral dissertation presented in January 1947 before the 
Academic Council of the Molotov Energy Institute in Moscow. In particular, see Part I of 
the book. This method was subsequently brought to fuller fruition in the classic book by 
Wozencraft and Jacobs (1965). Signal-space analysis is also discussed in Cioffi (1998), 
Anderson (1999), and Proakis (1995). 


2. In Section 5.7, we derived the union bound on the average probability of symbol error; the 
classic reference for this bound is Wozencraft and Jacobs (1965). For the derivation of tighter 
bounds, see Viterbi and Omura (1979, pp. 58-59). 


3. In Chapter 4, we used the following upper bound on the complementary error function 
exp(—#") 
Vru 


For large positive #, a second bound on the complementary error function is obtained by 
omitting the multiplying factor 1/u in the above upper bound, as shown by 


erfc(u) < 


erfc(u) < ant 


It is this second upper bound that is used in Equation (5.97). 
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5.1 In Section 3.7 we described line codes for pulse-code modulation. Referring to the materia] 
presented therein, formulate the signal constellations for the following line codes; 


(a) Unipolar nonreturn-to-zero code 
(b) Polar nonreturn-to-zero code 
(c) Unipolar return-to-zero code 
(d) Manchester code 
5.2 An 8-level PAM signal is defined by 


t 1 
s,(t) = A; a(t _ 5) 
where A; = +1, +3, +5, +7. Formulate the signal constellation of [sA 
5.3 Figure P5.3 displays the waveforms of four signals s4(ż), s2(t), s3(t), and s4(£). 
(a) Using the Gram-Schmidt orthogonalization procedure, find an orthonormal basis for 
this set of signals. 
(b) Construct the corresponding signal-space diagram. 


5) sol) 30) salt) 
1 1 Pee 1 
t t t t 
0 2 orm? 0 T 
3 3 


Ficure P5.3 


win 


5.4 (a) Using the Gram-Schmidt orthogonalization procedure, find a set of orthonormal basis 
functions to represent the three signals s,(£), s2(t), and s3(¢) shown in Figure P5.4. 
(b) Express each of these signals in terms of the set of basis functions found in part (a). 
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5.5 An orthogonal set of signals is characterized by the property that the inner produc 








FIGURE P5.4 





tof 


any pair of signals in the set is zero. Figure P5.5 shows a pair of signals s,(t) and soll) 
that satisfy this condition. Construct the signal constellation for s(t) and s(t). 
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Figure P5.5 


5.6 A source of information emits a set of symbols denoted by {m;}4;. Two candidate mod- 
ulation schemes, namely, pulse-duration modulation (PDM) and pulse-position modula- 
tion (PPM), are considered for the electrical representation of this set of symbols. In PDM, 
the ith symbol is represented by a pulse of unit amplitude and duration (i/M)T. On the 
other hand, in PPM, the ith symbol is represented by a short pulse of unit amplitude and 
fixed duration, which is transmitted at time t = (i/M)T. Show that PPM is the only one 
of the two that can produce an orthogonal set of signals over the interval 0 = ż = T. 

5.7 A set of 2M biorthogonal signals is obtained from a set of M orthogonal signals by 
augmenting it with the negative of each signal in the set. 

(a) The extension of orthogonal to biorthogonal signals leaves the dimensionality of the 
signal space unchanged. Why? 

(b) Construct the signal constellation for the biorthogonal signals corresponding to the 
pair of orthogonal signals shown in Figure P5.5. 

5.8 (a) A pair of signals s,(t) and s,(¢) have a common duration T. Show that the inner 

product of this pair of signals is given by 


T 
Í ,(t)s,(t)dt = sfs, 


where s; and s, are the vector representations of s,(t) and s,(t), respectively. 
(b) As a followup to part (a), show that 


z 
Í (s(t) — salt) dt = || s; — sa ||? 


5.9 Consider a pair of complex-valued signals s,(t) and s2(t) that are respectively represented 
by 
s(t) = 4y164(t) + ay2¢2(t), -o << 
s(t) = anġı(t) + aood2(t), -a<t<o 


where the basis functions $;(t) and $2(t) are both real valued, but the coefficients 411, 
442) 421, and a2 are complex valued. Prove the complex form of the Schwarz inequality: 


2 00 co. 
<f |si(#)|7de Í |s2(£) |*dt 


f: si(t)s2(¢)dt 


where the asterisk denotes complex conjugation. When is this relation satisfied with the 
equality sign? 








Random Processes 


5.10 Consider a random process X(t) expanded in the form 


N 
X(t) = È XO + Wo, O=tsT 
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where W’(t) is a remainder noise term. The {p(X form an orthonormal set over th 
interval 0 = t = T, and the X; are defined by : 


K 
X = Í X(t)ġ;(t)dt 
oO 


Let W’(t,) denote a random variable obtained by observing W'(t) at time t = t,. Show 
that 

EXW = Jalpan N 

[X; (t,)] = 0, 0<1,<T 


5.11 Consider the optimum detection of the sinusoidal signal 


. (87t 
t) = — ste 
s(t) saf T ), 0Ost=T 


in additive white Gaussian noise. 

(a) Determine the correlator output assuming a noiseless input. 

(b) Determine the corresponding matched filter output, assuming that the filter includes 
a delay T to make it causal. 

(c) Hence show that these two outputs are the same only at time instant t = T. 


Probability of Error 


5.12 Figure P5.12 shows a pair of signals s,(t) and sz(t) that are orthogonal to each other over 
the observation interval 0 = t = 3T. The received signal is defined by 

0<t<3T 

k=1,2 


where w(t) is white Gaussian noise of zero mean and power spectral density No/2. 

(a) Design a receiver that decides in favor of signals s,(¢) or s2(t), assuming that these 
two signals are equiprobable. 

(b) Calculate the average probability of symbol error incurred by this receiver for 
E/No = 4, where E is the signal energy. 


x(t) = s,(t) + w(t), 
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5.13 In the Manchester code, binary symbol 1 is represented by the doublet pulse s(t) show? 
in Figure P5.13, and binary symbol 0 is represented by the negative of this pulse. Denve 
the formula for the probability of error incurred by the maximum likelihood detector 
procedure applied to this form of signaling over an AWGN channel. 
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FIGURE P5.13 


5.14 Inthe Bayes test, applied to a binary hypothesis testing problem where we have to choose 
one of two possible hypotheses Hy and Hy, we minimize the risk R defined by 


R = CooPoP(say Ho|Ho is true) 
+ CyopoP(say H,|Ho is true) 
+ CypiP(say H, |H; is true) 
+ CoipiP(say Ho| H; is true) 
The terms Coo, C10; C11, and Co; denote the costs assigned to the four possible outcomes 
of the experiment: The first subscript indicates the hypothesis chosen, and the second the 


hypothesis that is true. Assume that Cy) > Coo and Co, > C,,. The po and p, denote the 
a priori probabilities of hypotheses Hy and H,, respectively. 


(a) Given the observation vector x, show that the partitioning of the observation space 
so as to minimize the risk R leads to the Jikelibood ratio test: 


say Hy if A(x) <A 
say H; if A(x) >A 


where A(x) is the likelihood ratio 


= fx(x|Hy) 
Fx(x| Ho) 


and A is the threshold of the test defined by 


ys Po(Cio — Coo) 
Pi(Cor — Cri) 


A(x) 


(b) What are the cost values for which the Bayes’ criterion reduces to the minimum 
probability of error criterion? 


Principles of Rotational and Translational Invariance 


5.15 Continuing with the four line codes considered in Problem 5.1, identify the line codes 
that have minimum average energy and those that do not. Compare your answers with 
the observations made on these line codes in Section 3.7. 


5.16 Consider the two constellations shown in Figure 5.11. Determine the orthonormal matrix 
Q that transforms the constellation shown in Figure 5.11a into the one shown in Figure 
5.11b. 
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5.17 (a) The two signal constellations shown in Figure P5.17 exhibit the same average Prob. 
ability of symbol error. Justify the validity of this statement. 


(b) Which of these two constellations has minimum average energy? Justify your angwe, 


You may assume that the symbols pertaining to the message points displayed in Figure 
P5.17 are equally likely. 
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5.18 Simplex (transorthogonal) signals are equally likely highly-correlated signals with the 
most negative correlation that can be achieved with a set of M orthogonal signals. That 
is, the correlation coefficient between any pair of signals in the set is defined by 

1 fori=j 
ao at for i # j 
One method of constructing simplex signals is to start with a set of M orthogonal signals, 
each with energy E, and then apply the minimum energy translate. 
Consider a set of three equally likely symbols whose signal constellation consists of 
the vertices of an equilateral triangle. Show that these three symbols constitute a simplex 
code. 


Bounds on Probability of Error 


5.19 In this problem we explore the approximations to the probability of an error, Pe, for the 
pair of antipodal signals shown in Figure P5.19 in the presence of additive white Gaussian 
noise of power spectral density No/2. The exact formula for P, is 


ae Es 
Pa = fete JE) 


(This formula is derived in Section 6.3) 

(a) Using the two upper bounds for the complementary error function given in Note 3, 
derive the corresponding approximations to P,. 

(b) Compare the approximations derived in part (a) for P, to the exact formula for 
E/N, = 9. For the exact calculation of P, you may use Table A6.6 on the error 
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5.20 Consider the special case of a signal constellation that has a symmetric geometry with 
respect to the origin. Assume that the M message points of the constellation, pertaining 
to symbols m, m2, ** + , my, are equally likely. Using the upper bound on the comple- 
mentary error function given in Equation (5.94), show that the average probability of 
symbol error for the constellation is bounded as 





p, < Moin exp | —min die 
Ve P| Me (ANo 


isk 


where d,, is the Euclidean distance between message points 7 and k, and M,,;, is the number 
of transmitted signals that attain the minimum Euclidean distance for each Mi 





PASSBAND DATA 
TRANSMISSION 


This chapter builds on the material developed in Chapter 5 on signal-space analysis. It 
discusses the subject of digital data transmission over a band-pass channel that can be 
linear or nonlinear. As with analog communications, this mode of data transmission relies 
on the use of a sinusoidal carrier wave modulated by the data stream. 

Specifically, the following topics are covered: 


P Different methods of digital modulation, namely, phase-shift keying, quadrature- 
amplitude modulation, and frequency-shift keying, and their individual variants. 


> Coherent detection of modulated signals in additive white Gaussian noise, which requires 
the receiver to be synchronized to the transmitter with respect to both carrier phase and 
bit timing. 


P Noncoherent detection of modulated signals in additive white Gaussian noise, 
disregarding phase information in the received signal. 


> Modems for the transmission and reception of digital data over the public switched 
telephone network. 


> Sophisticated modulation techniques, namely, carrierless amplitude/phase modulation and 
discrete multitone, for data transmission over a wideband channel with medium to severe 
intersymbol interference. 


» Techniques for synchronizing the receiver to the transmitter. 


i 6.1 Introduction 


In baseband pulse transmission, which we studied in Chapter 4, a data stream represented 
in the form of a discrete pulse-amplitude modulated (PAM) signal is transmitted directly 
over a low-pass channel. In digital passband transmission, on the other hand, the incoming 
data stream is modulated onto a carrier (usually sinusoidal) with fixed frequency limits 
imposed by a band-pass channel of interest; passband data transmission is studied in 
chapter. 

The communication channel used for passband data transmission may be a micto- 
wave radio link, a satellite channel, or the like. Yet other applications of passband data 
transmission are in the design of passband line codes for use on digital subscriber loops 
and orthogonal frequency-division multiplexing techniques for broadcasting. In any even, 
the modulation process making the transmission possible involves switching (keying) ™ 
amplitude, frequency, or phase of a sinusoida! carrier in some fashion in accordance W! 
the incoming data. Thus there are three basic signaling schemes, and they are known # 


344 


6.1 Introduction 345 























Binary 
data 0 1 1 QO 1 0 0 1 
l 
0 
m— T, — 
0 | t 
0 t 








to 


FIGURE 6.1 Illustrative waveforms for the three basic forms of signaling binary information. (a) 
Amplitude-shift keying. (b) Phase-shift keying. (c) Frequency-shift keying with continuous phase. 


amplitude-shift keying (ASK), frequency-shift keying (FSK), and phase-shift keying (PSK). 
They may be viewed as special cases of amplitude modulation, frequency modulation, and 
phase modulation, respectively. 

Figure 6.1 illustrates these three methods of modulation for the case of a source 
supplying binary data. The following points are noteworthy from Figure 6.1: 


> Although in continuous-wave modulation it is usually difficult to distinguish between 
phase-modulated and frequency-modulated signals by merely looking at their wave- 
forms, this is not true for PSK and FSK signals. 


> Unlike ASK signals, both PSK and FSK signals have a constant envelope. 


This latter property makes PSK and FSK signals impervious to amplitude nonlinearities, 
commonly encountered in microwave radio and satellite channels. It is for this reason, in 
practice, we find that PSK and FSK signals are preferred to ASK signals for passband data 
transmission over nonlinear channels. 


HIERARCHY OF DIGITAL MODULATION TECHNIQUES! 


Digital modulation techniques may be classified into coherent and noncoberent techniques, 
depending on whether the receiver is equipped with a phase-recovery circuit or not. The 
phase-recovery circuit ensures that the oscillator supplying the locally generated carrier 
wave in the receiver is synchronized (in both frequency and phase) to the oscillator sup- 
plying the carrier wave used to originally modulate the incoming data stream in the 
transmitter. 

As discussed in Chapter 4, in an M-ary signaling scheme, we may send any one of 
M possible signals s,(t), s2(t) ..., sm(t), during each signaling interval of duration T. For 
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almost all applications, the number of possible signals M = 2”, where n is an integer, Th, 
symbol duration T = nT», where T; is the bit duration. In passband data transmission 
these signals are generated by changing the amplitude, phase, or frequency of a sinusoidal 
carrier in M discrete steps. Thus we have M-ary ASK, M-ary PSK, and M-ary FSK digital 
modulation schemes. Another way of generating M-ary signals is to combine differen 
methods of modulation into a hybrid form. For example, we may combine discrete changes 
in both the amplitude and phase of a carrier to produce M-ary amplitude-phase keyin 
(APK). A special form of this hybrid modulation is M-ary quadrature-amplitude mod. 
ulation (QAM), which has some attractive properties. M-ary ASK is a special case of 
M-ary QAM. 

M-ary signaling schemes are preferred over binary signaling schemes for transmitting 
digital information over band-pass channels when the requirement is to conserve band. 
width at the expense of increased power. In practice, we rarely find a communication 
channel that has the exact bandwidth required for transmitting the output of an infor- 
mation source by means of binary signaling schemes. Thus when the bandwidth of the 
channel is less than the required value, we may use M-ary signaling schemes for maximum 
efficiency. To illustrate the bandwidth-conservation capability of M-ary signaling schemes 
consider the transmission of information consisting of a binary sequence with bit duration 
T,. If we were to transmit this information by means of binary PSK, for example, we 
would require a bandwidth that is inversely proportional to T,. However, if we take blocks 
of n bits and use an M-ary PSK scheme with M = 2” and symbol duration T = nT, the 
bandwidth required is proportional to 1/nT,,. This shows that the use of M-ary PSK enables 
a reduction in transmission bandwidth by the factor n = log. M over binary PSK. 

M-ary PSK and M-ary QAM are examples of linear modulation. However, they differ 
from each other in one important respect: An M-ary PSK signal has a constant envelope, 
whereas an M-ary QAM signal involves changes in the carrier amplitude. Accordingly, 
M-ary PSK can be used to transmit digital data over a nonlinear band-pass channel, 
whereas M-ary QAM requires the use of a linear channel. 

M-ary PSK, M-ary QAM, and M-ary FSK are commonly used in coherent systems. 
Amplitude-shift keying and frequency-shift keying lend themselves naturally to use in non- 
coherent systems whenever it is impractical to maintain carrier phase synchronization. But 
in the case of phase-shift keying, we cannot have “noncoherent PSK” because the term 
noncoherent means doing without carrier phase information. Instead, we employ a 
“pseudo PSK” technique known as differential phase-shift keying (DPSK), which (in a 
loose sense) may be viewed as the noncoherent form of PSK. In practice, M-ary FSK and 
M-ary DPSK are the commonly used forms of digital modulation in noncoherent systems. 


a PROBABILITY OF ERROR 


A major goal of passband data transmission systems is the optimum design of the receiver 
so as to minimize the average probability of symbol error in the presence of additive white 
Gaussian noise (AWGN). With this goal in mind, much of the material presented in this 
chapter builds on the signal-space analysis tools presented in Chapter 5. Specifically, # 
the study of each system we begin with the formulation of a signal constellation and 4 
construction of decision regions in accordance with maximum likelihood signal detection 
over an AWGN channel. These formulations set the stage for evaluating the probability 
of symbol error P,. Depending on the method of digital modulation under study, the 
evaluation of P, proceeds in one of two ways: 


> In the case of certain simple methods such as coherent binary PSK and coheret 
binary FSK, exact formulas are derived for P.. 
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> In the case of more elaborate methods such as coherent M-ary PSK and coherent 
M-ary FSK, we resort to the use of the union bound for deriving an approximate 
formula for P,. 


POWER SPECTRA 


To fully appreciate the practical virtues of different methods of digital modulation, we 
also need to study the power spectra of the resulting modulated signals. This latter issue 
is particularly important in two contexts: occupancy of the channel bandwidth and co- 
channel interference in multiplexed systems. i 

Given a modulated signal s(t), we may describe it in terms of its in-phase and quad- 
rature components as 


s(t) = s(t) cos(2af.T) — Solt) sin(2f,2) 
= Re[S(t) exp( j2zf.t)] 
where Re[-] is the real part of the expression contained inside the square brackets. We also 
have 


(6.1) 


S(t) = s(t) + jso(t) (6.2) 
and 
exp(j2af.T) = cos(2af.t) + j sin(27f.t) (6.3) 


The signal s(t) is the complex envelope (i.e., baseband version) of the modulated (band- 
pass) signal s(t). The components s;(t) and Sg(t) and therefore 5(t) are all low-pass signals. 
They are uniquely defined in terms of the band-pass signal s(t) and the carrier frequency 
fa» provided that the half-bandwidth of s(t) is less than the carrier frequency f.. 

Let Ss(f) denote the power spectral density of the complex envelope $(t). We refer 
to Ss(f) as the baseband power spectral density. The power spectral density, Ss(f), of the 
original band-pass signal s(t) is a frequency-shifted version of Ss(f), except for a scaling 
factor, as shown by 





Ss(t) = A [Solf — fe) + Salf + fal (6.4) 


It is therefore sufficient to evaluate the baseband power spectral density Sg(f). Since &(ż) 
is a low-pass signal, the calculation of Sp(f) should be simpler than the calculation of Ss(f). 
(See Example 1.7.) 


w BANDWIDTH EFFICIENCY 


Throughout this book we have emphasized that channel bandwidth and transmitted power 
constitute two primary “communication resources,” the efficient utilization of which pro- 
vides the motivation for the search for spectrally efficient schemes. The primary objective 
of spectrally efficient modulation is to maximize the bandwidth efficiency defined as the 
ratio of the data rate in bits per second to the effectively utilized channel bandwidth. A 
secondary objective is to achieve this bandwidth efficiency at a minimum practical expen- 
diture of average signal power or, equivalently, in a channel perturbed by additive white 
Gaussian noise, a minimum practical expenditure of average signal-to-noise ratio. 

With the data rate denoted by R, and the effectively used channel bandwidth by B, 
we may express the bandwidth efficiency, p, as 


p= = bits/s/Hz (6.5) 
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Recall from Chapter 4 that bandwidth efficiency is the product of two independent factors. 
one due to the possible use of multilevel encoding and the other due to spectral shaping 
In multilevel encoding, information transmission through the channel is carried out on the 
basis of blocks of bits rather than single bits. With efficient spectral shaping, bandwidth 
requirement on the channel is reduced by the use of pulse-shaping filters that smooth on 
the sharp transitions in the transmitted waveform. These two factors are therefore impor. 
tant in their own individual ways in determining the bandwidth efficiency of a passband 
data transmission system of interest. 


i 6.2 Passband Transmission Model 


In a functional sense, we may model a passband data transmission system as shown in 
Figure 6.2. First, there is assumed to exist a message source that emits one symbol every 
T seconds, with the symbols belonging to an alphabet of M symbols, which we denote by 
My, May - - - , My. The a priori probabilities P(m,), P(rt2),.- +> Pimm) specify the message 
source output. When the M symbols of the alphabet are equally likely, we write 


pi = Pim) 
(6.6) 


x for all i 


The M-ary output of the message source is presented to a signal transmission encoder, 
producing a corresponding vector s; made up of N real elements, one such set for each of 
the M symbols of the source alphabet; the dimension N is less than or equal to M. With 
the vector s; as input, the modulator then constructs a distinct signal s;(t) of duration T 
seconds as the representation of the symbol 7; generated by the message source. The signal 
s,(t) is necessarily an energy signal, as shown by 


2 
E; = | s?(t) dt, i=1,2,...,M 16.7) 


Norte that s;(t) is real valued. One such signal is transmitted every T seconds. The particular 
signal chosen for transmission depends in some fashion on the incoming message and 
possibly on the signals transmitted in preceding time slots. With a sinusoidal carrier, the 
feature that is used by the modulator to distinguish one signal from another is a step change 
in the amplitude, frequency, or phase of the carrier. (Sometimes, a hybrid form of mod- 
ulation that combines changes in both amplitude and phase or amplitude and frequency 
is used.) 
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FIGURE 6.2 Functional model of passband data transmission system. 
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Returning to the functional model of Figure 6.2, the bandpass communication chan- 
nel, coupling the transmitter to the receiver, is assumed to have two characteristics: 


1. The channel is linear, with a bandwidth that is wide enough to accommodate the 
transmission of the modulated signal s;(t) with negligible or no distortion. 

2. The channel noise w(t) is the sample function of a white Gaussian noise process of 
zero mean and power spectral density No/2. 


The assumptions made herein are basically the same as those invoked in Chapter 5 dealing 
with signal-space analysis. 

The receiver, which consists of a detector followed by a signal transmission decoder, 
performs two functions: 


1. It reverses the operations performed in the transmitter. 
2. It minimizes the effect of channel noise on the estimate 7 computed for the trans- 
mitted symbol 77;. 


i 6.3 Coherent Phase-Shift Keying 


With the background material on the coherent detection of signals in additive white Gaus- 
sian noise that was presented in Chapter 5 at our disposal, we are now ready to study 
specific passband data transmission systems. In this section we focus on coherent phase- 
shift keying (PSK) by considering binary PSK, QPSK and its variants, and finish up with 
M-ary PSK. 


@ BINARY PHASE-SHIFT KEYING 


Ina coherent binary PSK system, the pair of signals s;(t) and s(t) used to represent binary 
symbols 1 and 0, respectively, is defined by 


s(t) = = cos(af.t) (6.8) 
y Te 


s(t) = = cos(2af,t + n) = — E cos(2rrf.t) (6.9) 
b b 


where 0 = t < Tp, and E; is the transmitted signal energy per bit. To ensure that each 
transmitted bit contains an integral number of cycles of the carrier wave, the carrier fre- 
quency f- is chosen equal to 7,/T, for some fixed integer #,. A pair of sinusoidal waves 
that differ only in a relative phase-shift of 180 degrees, as defined in Eqautions (6.8) and 
(6.9), are referred to as antipodal signals. 

From this pair of equations it is clear that, in the case of binary PSK, there is only 
one basis function of unit energy, namely, 


ot) = iz cos(27f.t), 0st< T, (6.10) 
A 





Then we may express the transmitted signals s,(¢) and s,(t) in terms of ,(t) as follows: 
silt) = VE,oit), O=t<T, (6.11) 
and 


s(t) = -VE.bi(t), 05t<T, (6.12) 
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Ficure 6.3  Signal-space diagram for coherent binary PSK system. The waveforms depicting the 
transmitted signals s,(t) and s3(t), displayed in the inserts, assume # = 2, 


A coherent binary PSK system is therefore characterized by having a signal space 
that is one-dimensional (i.e., N = 1), with a signal constellation consisting of two message 
points (i.e., M = 2). The coordinates of the message points are 


Te 
Su = f $1(t)@ (t) dt 


(6.13) 
= +VE, 
and 
i 
= t)b,(t) dt 
S21 A salt)oal ) (6.14 


= -VE, 


The message point corresponding to s(t) is located at s11 = +VE,, and the message point 
corresponding to s2(t) is located at s21 = —V Es. Figure 6.3 displays the signal-space di- 
agram for binary PSK. This figure also includes two inserts, showing example waveforms 
of antipodal signals representing s(t) and s,(t). Note that the constellation of Figure 64 
has minimum average energy- 


Error Probability of Binary PSK 


To realize a rule for making a decision in favor of symbol 1 or symbol 0, we apply 
Equation (5.59) of Chapter 5. Specifically, we partition the signal space of Figure 6.3 into 
two regions: 

> The set of points closest to message point dat +V Es 
» The set of points closest to message point 2 at — V Ey. 
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This is accomplished by constructing the midpoint of the line joining these two message 
points, and then marking off the appropriate decision regions. In Figure 6.3 these decision 
regions are marked Z, and Z,, according to the message point around which they are 
constructed. 

The decision rule is now simply to decide that signal s,(£) (i.e., binary symbol 1) was 
transmitted if the received signal point falls in region Z}, and decide that signal s3(t) (i.e., 
binary symbol 0) was transmitted if the received signal point falls in region Zz. Two kinds 
of erroneous decisions may, however, be'made. Signal s,(t) is transmitted, but the noise is 
such that the received signal point falls inside region Z, and so the receiver decides in favor 
of signal s,(z). Alternatively, signal s,(t) is transmitted, but the noise is such that the re- 
ceived signal point falls inside region Z, and so the receiver decides in favor of signal s(t). 

To calculate the probability of making an error of the first kind, we note from Figure 
6.3 that the decision region associated with symbol 1 or signal s,(t) is described by 


Z0 <x <% 


where the observable element x; is related to the received signal x(t) by 


Th 
xy = | x(t) (t) dt (6.15) 
o 


The conditional probability density function of random variable X,, given that symbol 0 
[i.e., signal s2(ż)] was transmitted, is defined by 


1 
falil) = ae orl -5 is sa| 


1 1 
EA oo| -H (xı + vE;*| 


The conditional probability of the receiver deciding in favor of symbol 1, given that symbol 
0 was transmitted, is therefore 


(6.16) 





Pio -Í fx (x110) dx, 


i ge i (6.17) 
= V i an| - (x, + VE; | dx, 
Putting 
1 
ary, sf (xı + VE») (6.18) 


and changing the variable of integration from x; to z, we may rewrite Equation (6.1 7) in 
the compact form 


-4f 2 
Pu =y van Ol z”) dz 


aL ol Ee 
g No 


where erfc(+) is the complementary error function. 


(6.19) 
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Consider next an error of the second kind. We note that the signal space of Fj i 
6.3 is symmetric with respect to the origin. It follows therefore that Pox, the conditional 
probability of the receiver deciding in favor of symbol 0, given that symbol 1 was trans- 
mitted, also has the same value as in Equation (6.19). 

Thus, averaging the conditional error probabilities p10 and Por, we find that the 
average probability of symbol error or, equivalently, the bit error rate for coherent binary 
PSK is (assuming equiprobable symbols) i 


at [Ee 
P, = 5 ete{ =) (6.20} 


As we increase the transmitted signal energy per bit, E,, for a specified noise spectra 
density No, the message points corresponding to symbols 1 and 0 move further apart, and 
the average probability of error P, is correspondingly reduced in accordance with Equation 
(6.20), which is intuitively satisfying. 


Generation and Detection of Coherent Binary PSK Signals 


To generate a binary PSK signal, we see from Equations (6.8)-(6.10) that we have 
to represent the input binary sequence in polar form with symbols 1 and 0 represented by 
constant amplitude levels of +VE,, and —VE,, respectively. This signal transmission en- 
coding is performed by a polar nonreturn-to-zero (NRZ) level encoder. The resulting bi- 
nary wave and a sinusoidal carrier (t), whose frequency f. = (n./T,) for some fixed 
integer ,, are applied to a product modulator, as in Figure 6.42. The carrier and the 
timing pulses used to generate the binary wave are usually extracted from a common 
master clock. The desired PSK wave is obtained at the modulator output. 

To detect the original binary sequence of 1s and 0s, we apply the noisy PSK signal 
x(t) (at the channel output) to a correlator, which is.also supplied with a locally generated 
coherent reference signal ,(#), as in Figure 6.4b. The correlator output, x1, is compared 
with a threshold of zero volts. If x, > 0, the receiver decides in favor of symbol 1. On the 
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Ficore 6.4 Block diagrams for (a) binary PSK transmitter and (b) coherent binary PSK 
receiver. 


6.3 Coherent Phase-Shift Keying 353 


other hand, if x, < 0, it decides in favor of symbol 0. If x, is exactly zero, the receiver 
makes a random guess in favor of 0 or 1. 


Power Spectra of Binary PSK Signals 


From the modulator of Figure 6.44, we see that the complex envelope of a binary 
PSK wave consists of an in-phase component only. Furthermore, depending on whether 
we have symbol 1 or symbol 0 at the modulator input during the signaling interval 
0 = t < Ta, we find that this in-phase component equals +g(t) or —g(t), respectively, 
where g(t) is the symbol shaping function defined by 


2E, ; 
giy’ PSIS (6.21) 


0, otherwise 


We assume that the input binary wave is random, with symbols 1 and 0 equally likely and 
the symbols transmitted during the different time slots being statistically independent. In 
Example 1.6 of Chapter 1 it is shown that the power spectral density of a random binary 
wave so described is equal to the energy spectral density of the symbol shaping function 
divided by the symbol duration. The energy spectral density of a Fourier transformable 
signal g(t) is defined as the squared magnitude of the signal’s Fourier transform. Hence, 
the baseband power spectral density of a binary PSK signal equals 


_ 2E, sin*(aT,f) 
(nT. f? (6.22) 
= 2E, sinc?(T,f) 
This power spectrum falls off as the inverse square of frequency, as shown in Figure 6.5. 
Figure 6.5 also includes a plot of the baseband power spectral density of a binary 


_ FSK signal, details of which are presented in Section 6.5. Comparison of these two spectra 
is deferred to that section. 
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Freure 6.5 Power spectra of binary PSK and FSK signals. 
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E QUADRIPHASE-SHIFT KEYING 


The provision of reliable performance, exemplified by a very low probability of error i" 
one important goal in the design of a digital communication system. Another important 
goal is the efficient utilization of channel bandwidth. In this subsection, we study a bang. 
width-conserving modulation scheme known as coherent quadriphase-shift keying, which 
is an example of quadrature-carrier multiplexing. 

In quadriphase-shift keying (QPSK), as with binary PSK, information carried by the 
transmitted signal is contained in the phase. In particular, the phase of the carrier takes 
on one of four equally spaced values, such as 7/4, 377/4, 57/4, and 77/4. For this set of 
values we may define the transmitted signal as 


2E 3 T ee 
s(t) = E cos| 2 + (2i — 1) z) 0stsT (6.23) 


0, elsewhere 


where i = 1, 2, 3, 4; E is the transmitted signal energy per symbol, and T is rhe symbol 
duration. The carrier frequency f. equals ,/T for some fixed integer n. Each possible 
value of the phase corresponds to a unique dibit. Thus, for example, we may choose the 
foregoing set of phase values to represent the Gray-encoded set of dibits: 10, 00, 01, and 
11, where only a single bit is changed from one dibit to the next. 


Signal-Space Diagram of QPSK 


Using a well-known trigonometric identity, we may use Equation (6.23) to redefine 
the transmitted signal s,(t) for the interval 0 < ¢ = T in the equivalent form: 


s(t) = Bola- H cos(2mf.t) — LOCE H sin(2af.t) (6.24) 


where i = 1, 2, 3, 4. Based on this representation, we can make the following observations: 


> There are two orthonormal basis functions, (t) and ¢»(t), contained in the expan- 
sion of s;(ż). Specifically, # ,(f) and ¢2(t) are defined by a pair of quadrature carriers: 


$t) = fento, OstsT (6.25) 
b(t) = G sin(2rf-t), 0O<t<T (6.26) 


TABLE 6.1 Signal-space 
characterization of QPSK 


Coordinates of 





Phase of Message Points 
Gray-encoded QPSK Signal §=——W—+—_ 
Input Dibit (radians) Si Si2 
10 al4 +VE/2 —VE/2 
00 37/4 -VER ~VE/2 
01 5l4 —VE/2 +VE/2 


11 7rl4 +V E/2 +VE/2 
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FIGURE 6.6 Signal-space diagram of coherent QPSK system. 


» There are four message points, and the associated signal vectors are defined by 


VE cos( 2 - 1) z) 


s; = i= 1,2,3,4 (6.27) 


-VE sin 2 - 1) 7) 
The elements of the signal vectors, namely, s; and sz, have their values summarized 
in Table 6.1. The first two columns of this table give the associated dibit and phase 
of the QPSK signal. 


Accordingly, a QPSK signal has a two-dimensional signal constellation (i.e., N = 2) 


and 


four message points (i.e., M = 4) whose phase angles increase in a counterclockwise di- 
rection, as illustrated in Figure 6.6. As with binary PSK, the QPSK signal has minimum 


average energy. 


EXAMPLE 6.1 


Figure 6.7 illustrates the sequences and waveforms involved in the generation of a QPSK 
signal, The input binary sequence 01101000 is shown in Figure 6.7a. This sequence is divided 
into two other sequences, consisting of odd- and even-numbered bits of the input sequence. 
These two sequences are shown in the top lines of Figures 6.7b and 6.7c. The waveforms 
representing the two components of the QPSK signal, namely, s;,¢,(¢) and s;22(t), are also 
shown in Figures 6.7b and 6.7c, respectively. These two waveforms may individually be 
viewed as examples of a binary PSK signal. Adding them, we get the QPSK waveform shown 
in Figure 6.7d. 

To define the decision rule for the detection of the transmitted data sequence, we par- 
tition the signal space into four regions, in accordance with Equation (5.59) of Chapter 5. 
The individual regions are defined by the set of points closest to the message point represented 
by signal vectors s1, $2, 83, and s4. This is readily accomplished by constructing the perpen- 
dicular bisectors of the square formed by joining the four message points and then marking 
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FIGURE 6.7 (a) Input binary sequence. (b) Odd-numbered bits of inputsequence and associated 
binary PSK wave. (c) Even-numbered bits of input sequence and associated binary PSK wave. 
(d) QPSK waveform defined as s(t) = sah (t) + s:2%2(t). 


off the appropriate regions. We thus find that the decision regions are quadrants whose vertices 
coincide with the origin. These regions are marked Z1, Z2, Z3, and Z4, in Figure 6.6, according 
to the message point around which they are constructed. < 


Error Probability of QPSK 
In a coherent QPSK system, the received signal x(t) is defined by 


0<t=T 


(6.28) 
i= 1,2, 3, 4 


x(t) = s,(t) + w(t), { 
where w(t) is the sample function of a white Gaussian noise process of zero mean and 


power spectral density No/2. Correspondingly, the observation vector x has two elements, 
x, and x2, defined by 


n 
x= |, swt at 
= VE cosl (2 - 1) z] + w (6.29) 


+ E+ 
= 3 1 


il 
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and 


T 
x= j, albat dt 


0 a 


Hl 


-VE sinl o -1) z] + wy (6.30) 


JE 
= Ete 


Thus the observable elements x, and x2 are sample values of independent Gaussian random 
variables with mean values equal to +V E/2 and +V E/2, respectively, and with a common 
variance equal to No/2. 

The decision rule is now simply to decide that s;(t) was transmitted if the received 
signal point associated with the observation vector x falls inside region Z,, decide that 
s2(t) was transmitted if the received signal point-falls inside region Zz, and so on. An 
erroneous decision will be made if, for example, signal s,(t) is transmitted but the noise 
w(t) is such that the received signal point falls outside region Z4. 

To calculate the average probability of symbol error, we note from Equation (6.24) 
that a coherent QPSK system is in fact equivalent to two coherent binary PSK systems 
working in parallel and using two carriers that are in phase quadrature; this is merely a 
statement of the quadrature-carrier multiplexing property of coherent QPSK. The in-phase 
channel output x; and the quadrature channel output x2 (i.e., the two elements of the 
observation vector x) may be viewed as the individual outputs of the two coherent binary 
PSK systems. Thus, according to Equations (6.29) and (6.30), these two binary PSK sys- 
tems may be characterized as follows: 


il 


» The signal energy per bit is E/2. 
> The noise spectral density is No/2. 


Hence, using Equation (6.20) for the average probability of bit error of a coherent binary 
PSK system, we may now state that the average probability of bit error in each channel of 
the coherent QPSK system is 


(6.31) 





Another important point to note is that the bit errors in the in-phase and quadrature 
channels of the coherent QPSK system are statistically independent. The in-phase channel 
makes a decision on one of the two bits constituting a symbol (dibit) of the QPSK signal, 
and the quadrature channel takes care of the other bit. Accordingly, the average probability 
of a correct decision resulting from the combined action of the two channels working 


together is 
2 
1 E 
= £ a3 ere x)| (6.32) 
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The average probability of symbol error for coherent QPSK is therefore 
P, =1-P, 


TE 1 if E (6.33) 
= ete{ =) 4 erfc ( 5) 


In the region where (E/2No) >> 1, we may ignore the quadratic term on the right-hang 
side of Equation (6.33), so we approximate the formula for the average probability of 
symbol error for coherent QPSK as 


LE 
P, = ete{ =) (6.34) 


The formula of Equation (6.34) may also be derived in another insightful way, using 
the signal-space diagram of Figure 6.6. Since the four message points of this diagram are 
circularly symmetric with respect to the origin, we may apply Equation (5.92), reproduced 
here in the form 








1 dix ; 
Ps 3 >» erte( 74) for all i (6.35) 
k+i 


Consider, for example, message point m, (corresponding to dibit 10) chosen as the trans- 
mitted message point. The message points m, and m4 (corresponding to dibits 00 and 11} 
are the closest to m,. From Figure 6.6 we readily find that 77, is equidistant from m, and 
mz, in a Euclidean sense, as shown by 


dy. = dy, = V2E 


Assuming that E/No is large enough to ignore the contribution of the most distant message 
point #13 (corresponding to dibit 01) relative to m, we find that the use of Equation (6.35) 
yields an approximate expression for P, that is the same as Equation (6.34). Note that in 
mistaking either 7 or m4 for my, a single bit error is made; on the other hand, in mistaking 
m; for m, two bit errors are made. For a high enough E/No, the likelihood of both bits 
of a symbol being in error is much less than a single bit, which is a further justification 
for ignoring 7 in calculating P, when m, is sent. 

In a QPSK system, we note that since there are two bits per symbol, the transmitted 
signal energy per symbol is twice the signal energy pet bit, as shown by 


E=2E, (6.36) 


Thus expressing the average probability of symbol error in terms of the ratio E,/No, we 


may write 
Po = ere{ Fe) (6.37) 


With Gray encoding used for the incoming symbols, we find from Equations (6.31) 
and (6.36) that the bit error rate of QPSK is exactly 


1 E, 
== — 6.38) 


We may therefore state that a coherent QPSK system achieves the same average probability 
of bit error as a coherent binary PSK system for the same bit rate and the same E,/No 
but uses only half the channel bandwidth. Stated in a different way, for the same E,INo 
and therefore the same average probability of bit error, a coherent QPSK system transmits 
information at twice the bit rate of a coherent binary PSK system for the same channe 
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bandwidth. For a prescribed performance, QPSK uses channel bandwidth better than bi- 
nary PSK, which explains the preferred use of QPSK over binary PSK in practice. 


Generation and Detection of Coherent QPSK Signals 


Consider next the generation and detection of QPSK signals. Figure 6.84 shows a 
block diagram of a typical QPSK transmitter. The incoming binary data sequence is first 
transformed into polar form by a nonreturn-to-zera level encoder. Thus, symbols 1 and 0 
are represented by +VE, and —VE,, respectively. This binary wave is next divided by 
means of a demultiplexer into two separate binary waves consisting of the odd- and even- 
numbered input bits. These two binary waves are denoted by a(t) and a2(t). We note that 
in any signaling interval, the amplitudes of a,(¢) and a(t) equal s4 and sj, respectively, 
depending on the particular dibit that is being transmitted. The two binary waves a(t) 
and a(t) are used to modulate a pair of quadrature carriers or orthonormal basis functions: 
(t) equal to V2/T cos(27f,t) and (t) equal to V2/T sin(27f,t). The result is a pair of 
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FIGURE 6.8 Block diagrams of (a) QPSK transmitter and (b) coherent QPSK receiver. 
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binary PSK signals, which may be detected independently due to the orthogonality of CR 
and ¢,(t). Finally, the two binary PSK signals are added to produce the desired QPSK 
signal. 

The QPSK receiver consists of a pair of correlators with a common input and supplied 
with a locally generated pair of coherent reference signals ,(¢) and ¢2(t), as in Figure 
6.86. The correlator outputs x, and x2, produced in response to the received signal x(t) 
are each compared with a threshold of zero. If x, > 0, a decision is made in favor of 
symbol 1 for the in-phase channel output, but if x; < 0, a decision is made in favor of 
symbol 0. Similarly, if x. > 0, a decision is made in favor of symbol 1 for the quadrature 
channel output, but if x, < 0, a decision is made in favor of symbol 0. Finally, these two 
binary sequences at the in-phase and quadrature channel outputs are combined in a myj. 
tiplexer to reproduce the original binary sequence at the transmitter input with the minj. 
mum probability of symbol error in an AWGN channel. 


Power Spectra of QPSK Signals 


Assume that the binary wave at the modulator input is random, with symbols 1 and 
0 being equally likely, and with the symbols transmitted during adjacent time slots being 
statistically independent. We make the following observations pertaining to the in-phase 
and quadrature components of a QPSK signal: 


1. Depending on the dibit sent during the signaling interval -T,, = t = T,, the in-phase 
component equals +g(t) or —g(t), and similarly for the quadrature component. The 
g(t) denotes the symbol shaping function, defined by 


E 
a(t) = E ree (6.39) 


0, otherwise 


Hence, the in-phase and quadrature components have a common power speciral 
density, namely, E sinc*(Tf). 


1.0 


Normalized power spectral density, S,(f/4E, 
[=] 
a 


> 
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FIGURE 6.9 Power spectra of QPSK and MSK signals. 
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2. The in-phase and quadrature components are statistically independent. Accordingly, 
the baseband power spectral density of the QPSK signal equals the sum of the indi- 
vidual power spectral densities of the in-phase and quadrature components, so we 
may write 

Salf) = 2E sinc?(Tf) 
= 4E, sinc?(2T,f) 
Figure 6.9 plots Ss(f), normalized with respect to 4E,, versus the normalized fre- 
quency fT,. This figure also includes a plot of the baseband power spectral density of a 


certain form of binary FSK called minimum shift keying, the evaluation of which is pre- 
sented in Section 6.5. Comparison of these two spectra is deferred to that section. 


(6.40) 


m OFFSET QPSK 


The signal space diagram of Figure 6.10a embodies all the possible phase transitions that 
can arise in the generation of a QPSK signal. More specifically, examining the QPSK 
waveform illustrated in Figure 6.7 for Example 6.1, we may make the following 
observations: 


1. The carrier phase changes by +180 degrees whenever both the in-phase and quad- 
rature components of the QPSK signal changes sign. An example of this situation is 
illustrated in Figure 6.7 when the input binary sequence switches from dibit 01 to 
dibit 10. 

2. The carrier phase changes by +90 degrees whenever the in-phase or quadrature 
component changes sign. An example of this second situation is illustrated in Figure 
6.7 when the input binary sequence switches from dibit 10 to dibit 00, during which 
the in-phase component changes sign, whereas the quadrature component is 
unchanged. 

3. The carrier phase is unchanged when neither the in-phase component nor the quad- 
rature component changes sign. This last situation is illustrated in Figure 6.7 when 
dibit 10 is transmitted in two successive symbol intervals. 


Situation 1 and, to a much lesser extent, situation 2 can be of a particular concern when 
the QPSK signal is filtered during the course of transmission, prior to detection. Specifi- 
cally, the 180- and 90-degree shifts in carrier phase can result in changes in the carrier 
amplitude.(i.e., envelope of the QPSK signal), thereby causing additional symbol errors on 
detection. 
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Ficure 6.10 Possible paths for switching between the message points in (a) QPSK and 
(b) offset QPSK. 
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The extent of amplitude fluctuations exhibited by QPSK signals may be teduced } 
using offset OPSK.? In this variant of QPSK, the bit stream responsible for generating t 
quadrature component is delayed (i.e., offset) by half a symbol interval with respect to the 
bit stream responsible for generating the in-phase component. Specifically, the two basis 
functions of offset QPSK are defined by 


2 

(t) = E cos(27ft), OstSsT (6.41) 
2 T 3T 

$2(t) = E sin(27f,t), 5 at (6.42) 


Accordingly, unlike QPSK, the phase transitions likely to occur in offset QPSK are confined 
to +90 degrees, as indicated in the signal space diagram of Figure 6.10. However, +99 
degree phase transitions in offset QPSK occur twice as frequently but with half the intensity 
encountered in QPSK. Since, in addition to +90-degree phase transitions, +180-degree 
phase transitions also occur in QPSK, we find that amplitude fluctuations in offset QPSK 
due to filtering have a smaller amplitude than in the case of QPSK. 

Despite the delay T/2 applied to the basis function ġ2(t) in Equation (6.42) compared 
to that in Equation (6.26), the offset QPSK has exactly the same probability of symbol 
error in an AWGN channel as QPSK. The equivalence in noise performance between these 
phase-shift keying schemes assumes the use of coherent detection. The reason for the equiv- 
alence is that the statistical independence of the in-phase and quadrature components 
applies to both QPSK and offset QPSK. We may therefore say that the error probability 
in the in-phase or quadrature channel of a coherent offset QPSK receiver is still equal to 
(1/2) erfc(V E/2No). Hence the formula of Equation (6.34) applies equally well to the offset 
QPSK. 


8 77/4-SHIFTED QPSK 


An ordinary QPSK signal may reside in either one of the two commonly used constellations 
shown in Figures 6.11¢ and 6.115, which are shifted by 77/4 radians with respect to each 
other. In another variant of QPSK known as 77/4-shifted QPSK, the carrier phase used 
for the transmission of successive symbols (i.e., dibits) is alternately picked from one of 
the two QPSK constellations in Figure 6.11 and then the other. It follows therefore that a 
a/4-shifted QPSK signal may reside in any one of eight possible phase states, as indicated 
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FiGure 6.11 Two commonly used signal constellations for QPSK; the arrows indicate the paths 
along which the QPSK modulator can change its state. 
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FIGURE 6.12 Eight possible phase states for the 7/4-shifted QPSK modulator. 


in Figure 6.12. The four dashed lines emanating from each possible message point in 
Figure 6.12 define the phase transitions that are feasible in 7/4-shifted QPSK. 

Table 6.2 summarizes a possible set of relationships between the phase transitions 
in this new digital modulation scheme and the incoming Gray-encoded dibits. For example, 
if the modulator is in one of the phase states portrayed in Figure 6.116, then on receiving 
the dibit 00 it shifts into a phase state portrayed in Figure 6.114 by rotating through 7/4 
radians in a counterclockwise direction. 

Attractive features of the 7r/4-shifted QPSK scheme include the following: 


* The phase transitions from one symbol to the next are restricted to +7/4 and +37/4 
radians, which is to be contrasted with the +77/2 and +7 phase transitions in QPSK. 
Consequently, envelope variations of 7/4-shifted QPSK signals due to filtering are 
significantly reduced, compared to those in QPSK. 

> Unlike offset QPSK signals, 7/4-shifted QPSK signals can be noncoherently detected, 
thereby considerably simplifying the receiver design. Moreover, like QPSK signals, 
a/4-shifted QPSK can be differently encoded, in which case we should really speak 
of 7/4-shifted DOPSK. 


The generation of 77/4-shifted DQPSK symbols, represented by the symbol pair (I, Q), 
is described by the following pair of relationships (see Problem 6.13): 


I, = cos(6,_; + Aba) 


(6.43) 
= cos & 
Q, = sin(O,—1 + Ag) (6.44) 
= sin & 







| TABLE 6.2 Correspondence between input 
dibit and phase change for 1/4-shifted 
| DQOPSK 


Gray-Encoded Input Dibit Phase Change, A® (radians) 








00 al4 
01 3a/4 
11 —30/4 


10 —al4 
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į TABLE 6.3 1/4-shifted DQPSK results for Example 6.2 





Phase 0-1 Phase Change AQ, Transmitted Phase 4, 
Step k (radians) Input Dibit (radians) (radians) 
1 al4 00 al4 mi2 
2 mi2 10 —al4 a4 
3 a4 10 —al4 0 
4 0 01 3a/4 30/4 





where 6,_, is the absolute phase angle of symbol k — 1, and A& is the differentially 
encoded phase change defined in accordance with Table 6.2. 


p EXAMPLE 6.2 


Continuing with the input binary sequence of Example 6.1, namely, 01101000, suppose that 
the phase angle 6) = 7/4 in the constellation of Figure 6.116 is assigned as the initial phase 
state of the 7/4-shifted DQPSK modulator. Then, arranging the input binary sequence as a 
sequence of dibits and following the convention of Table 6.2, we get the results presented in 
Table 6.3 for the example at hand. 4 


Detection of «/4-Shifted DQPSK Signals 


Having familiarized ourselves with the generation of 7/4-shifted DQPSK signals, we 
go on to consider their differential detection. Given the noisy channel output x(t), the 
receiver first computes the projections of x(#) onto the basis functions ¢ ,(t) and @,(t). The 
resulting outputs, denoted by I and Q, respectively, are applied to a differential detector 
that consists of the following components, as indicated in Figure 6.13: 


> Arctangent computer for extracting the phase angle 6 of the channel output (received 
signal). 

> Phase-difference computer for determining the change in the phase 6 occurring over 
one symbol interval. 

> Modulo-27 correction logic for correcting errors due to the possibility of phase angles 
wrapping around the real axis. 


Elaborating further on the latter point, let A@, denote the computed phase difference be- 
tween 0, and 6,_, representing the phase angles of the channel output for symbols k and 
k — 1, respectively. Then the modulo-27 correction logic operates as follows: 


IF A@, < —180 degrees THEN A0, = AQ, + 360 degrees 6.45) 
IF A8, > 180 degrees THEN A@, = AQ, — 360 degrees j 


Phase-difference 
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FIGURE 6.13 Block diagram of the 77/4-shifted DQPSK detector. 
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FIGURE 6.14 Illustrating the possibility of phase angles wrapping around the positive real axis. 


To illustrate the need for this phase correction, consider the situation depicted in Figure 
6.14, where 6,1 = 350 degrees and 6, = 60 degrees, both phase angles being measured 
in a counterclockwise direction. From this figure we readily see that the phase change AO, 
measured in a counterclockwise direction, is 70 degrees. However, without correction the 
phase change AQ, is computed as 60 degrees — 350 degrees = —290 degrees. Applying 
the first line of Equation (6.45), the modulo-2 7 correction logic compensates for the wrap- 
around the positive real axis, yielding the corrected result 


A0, = —290 degrees + 360 degrees = 70 degrees 


The tangent type differential detector of Figure 6.13 for the demodulation of 17/4- 
shifted DQPSK signals is relatively simple to implement. It offers a satisfactory perfor- 
mance in a Rayleigh fading channel as in a static multipath environment. However, when 
the multipath environment is time varying as experienced in a commercial digital wireless 
communication system, computer simulation results appear to show that the receiver per- 
formance degrades very rapidly.* 


= M-ary PSK 


QPSK is a special case of M-ary PSK, where the phase of the carrier takes on one of M 
possible values, namely, 6, = 2(i — 1)a/M, where i = 1, 2,..., M. Accordingly, during 
each signaling interval of duration T, one of the M possible signals 


2E la 
(t)= [= + = (i- i= sects : 
5;(t) T cos( 2ft M {i n), i= 1,2, M (6.46) 


is sent, where E is the signal energy per symbol. The carrier frequency f. = ,/T for some 
fixed integer n,. 

Each s;(t) may be expanded in terms of the same two basis functions ¢ (t) and ¢(t) 
defined in Equations (6.25) and (6.26), respectively. The signal constellation of M-ary PSK 
is therefore two-dimensional. The M message points are equally spaced on a circle of radius 
VE and center at the origin, as illustrated in Figure 6.154, for the case of octaphase- 
shift-keying (i.e., M = 8). 

From Figure 6.154 we note that the signal-space diagram is circularly symmetric. We 
may therefore apply Equation (5.92), based on the union bound, to develop an approxi- 
mate formula for the average probability of symbol error for M-ary PSK. Suppose that the 
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FIGURE 6.15 (a) Signal-space diagram for octaphase-shift keying (i.e., M = 8). The decision 
boundaries are shown as dashed lines. (b) Signal-space diagram illustrating the application of the 
union bound for octaphase-shift keying. 
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transmitted signal corresponds to the message point mı, whose coordinates along the 
$1- and ¢ y-axes are +-VE and 0, respectively. Suppose that the ratio E/N is large enough 
to consider the nearest two message points, one on either side of m4, as potential candidates 
for being mistaken for 7, due to channel noise. This is illustrated in Figure 6.156 for the 
case of M = 8. The Euclidean distance of each of these two points from my, is (for M = 8) 


di2 = dig = 2VE sin( 2) 


Hence, the use of Equation (5.92) of Chapter 5 yields the average probability of symbol 
error for coherent M-ary PSK as 


P, = ore JE sin( 2) (6.47) 


where it is assumed that M = 4, The approximation becomes extremely tight, for fixed 
M, as E/N is increased. For M = 4, Equation (6.47) reduces to the same form given in 
Equation (6.34) for QPSK. 


Power Spectra of M-ary PSK Signals 
The symbol duration of M-ary PSK is defined by 


T = T, log: M (6.48) 


where T; is the bit duration. Proceeding in a manner similar to that described for a QPSK 
signal, we may show that the baseband power spectral density of an M-ary PSK signal is 
given by 


Ss(f) = 2E sinc?(Tf) 


49 
= 2E, log, M sinc?(T;f log, M) (6:43) 


In Figure 6.16, we show the normalized power spectral density Sg(f)/2E, plotted versus 
the normalized frequency fT, for three different values of M, namely, M = 2, 4, 8. 
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Figure 6.16 Power spectra of M-ary PSK signals for M = 2, 4, 8. 
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TABLE 6.4 Bandwidth efficiency of 
M-ary PSK signals 


M 2 4 8 16 32 64 








p (bits/s/Hz) 0.5 1 1.5 2 2.5 3 





BANDWIDTH EFFICIENCY OF M-ary PSK SIGNALS 


The power spectra of M-ary PSK signals possess a main lobe bounded by well-defined 
spectral nulls (i.e., frequencies at which the power spectral density is zero). Accordingly 
the spectral width of the main lobe provides a simple and popular measure for the band. 
width of M-ary PSK signals. This definition is referred to as the null-to-null bandwidth 
With the null-to-null bandwidth encompassing the main lobe of the power spectrum of an 
M-ary signal, we find that it contains most of the signal power. This is readily seen by 
looking at the power spectral plots of Figure 6.16. 

For the passband basis functions defined in Equations (6.25) and (6.26), the channel 
bandwidth required to pass M-ary PSK signals (more precisely, the main spectral lobe of 
M-ary signals) is given by 


B= (6.50) 


2 
T 
where T is the symbol duration. But the symbol duration T is related to the bit duration 


T, by Equation (6.48). Moreover, the bit rate R, = 1/T,. Hence, we may redefine the 
channel bandwidth of Equation (6.50) in terms of the bit rate R, as 


O 2R 
log, M 
Based on this formula, the bandwidth efficiency of M-ary PSK signals is given. by 
-Èe 
B 


_ log, M 
2 





(6.51) 


p 
(6.52) 


Table 6.4 gives the values of p calculated from Equation (6.52) for varying M. 

In light of Equation (6.47) and Table 6.4, we make the following observation in the 
context of M-ary PSK: As the number of states, M, is increased, the bandwidth efficiency 
is improved at the expense of error performance. To ensure that there is no degradation 
in error performance, we have to increase E,/Np to compensate for the increase in M. 


6.4 Hybrid Amplitude/Phase 
Modulation Schemes 














In an M-ary PSK system, the in-phase and quadrature components of the modulated signal 
are interrelated in such a way that the envelope is constrained to remain constant. This 
constraint manifests itself in a circular constellation for the message points. However: 
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this constraint is removed, and the in-phase and quadrature components are thereby per- 
mitted to be independent, we get a new modulation scheme called M-ary quadrature am- 
plitude modulation (QAM). This latter modulation scheme is hybrid in nature in that the 
carrier experiences amplitude as well as phase modulation. 

The passband basis functions in M-ary QAM may not be periodic for an arbitrary 
choice of the carrier frequency f, with respect to the symbol rate 1/T. Ordinarily, this 
aperiodicity is of no real concern. By reformulating the expression for the transmitted 
signal in a certain way, it is possible to eliminate the time variation of the basis functions 
on successive symbol transmissions, and-thereby simplify implementation of the transmit- 
ter. In particular, the transmitter is made to appear “‘carrierless,”’ while fully retaining the 
essence of the hybridized amplitude and phase modulation process. This is indeed the idea 
behind the carrierless amplitude/phase modulation (CAP). 

Despite the differences between QAM and CAP in their implementation details, they 
have exactly the same signal constellations. Accordingly, they are fundamentally equivalent 
in performance for a prescribed receiver complexity. In what follows we first discuss QAM 
and then CAP. ; 


m M-ARY QUADRATURE AMPLITUDE MODULATION 


In Chapters 4 and 5, we studied M-ary pulse amplitude modulation (PAM), which is one- 
dimensional. M-ary QAM is a two-dimensional generalization of M-ary PAM in that its 
formulation involves two orthogonal passband basis functions, as shown by 


e(t) = ie cos(27f-t), OstsT (6.53) 
(t) = E sin(2 rft), 0<t<T (6.54) 


Let the ith message point s; in the (@1, #2) plane be denoted by (a;dmin/2, Bid min/2), where 
din is the minimum distance between any two message points in the constellation, a; and 
b; are integers, and i = 1, 2, . . . , M. Let (dmin/2) = VEo, where Eg is the energy of the 
signal with the lowest amplitude. The transmitted M-ary QAM signal for symbol k, say, 
is then defined by 


2E 2E, : OstsT 
selt) = JEE ag cost2aft) = AF be sin(2nf.1), kenai inne O 


The signal s; (t) consists of two phase-quadrature carriers with each one being modulated 
by a set of discrete amplitudes, hence the name quadrature amplitude modulation. 

Depending on the number of possible symbols M, we may distinguish two distinct 
QAM constellations: square constellations for which the number of bits per symbol is 
even, and cross constellations for which the number of bits per symbol is odd. These two 
cases are considered in the sequel in that order. 


QAM Square Constellations 
With an even number of bits per symbol, we may write 
L=VM (6.56) 


where L is a positive integer. Under this condition, an M-ary QAM square constellation 
can always be viewed as the Cartesian product of a one-dimensional L-ary PAM constel- 
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lation with itself. By definition, the Cartesian product of two sets of coordinates (repre. 
senting a pair of one-dimensional constellations) is made up of the set of all Possible 
ordered pairs of coordinates with the first coordinate in each such pair taken from the firs 
set involved in the product and the second coordinate taken from the second set in the 
product. 


In the case of a QAM square constellation, the ordered pairs of coordinates naturally 


form a square matrix, as shown by 


(CL+1,L-1) (-L+3,L-1) ~ (L-1,L-1) 


=b44, b= -L+ =I a (he he 
ie (-L > 3) ( 3 L-3) ( , 3) 
(-L+1,-L4+1) (-L+3,-L+1) .. (L-1,-L+1) 

(6.57) 


P> EXAMPLE 6.3 


Consider a 16-QAM whose signal constellation is depicted in Figure 6.174. The encoding of 
the message points shown in this figure is as follows: 


> Two of the four bits, namely, the left-most two bits, specify the quadrant in the ($,, »)- 
plane in which a message point lies. Thus, starting from the first quadrant and pro- 
ceeding counterclockwise, the four quadrants are represented by the dibits 11, 10, 00, 
and 01. 

» The remaining two hits are used to represent one of the four possihle symbols lying 
within each quadrant of the (#1, #2)-plane. 


Note that the encoding of the four quadrants and also the encoding of the symbols in each 
quadrant follow the Gray coding rule. 
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FIGURE 6.17 (a) Signal-space diagram of M-ary QAM for M = 16; the message points in each 
quadrant are identified with Gray-encoded quadbits. (b) Signal-space diagram of the correspond- 
ing 4-PAM signal. 
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For the example at hand, we have L = 4. Thus the square constellation of Figure 6.174 
is the Cartesian product of the 4-PAM constellation shown in Figure 6.17b with itself. More- 
over, the matrix of Equation (6.57) has the value 


(3,3) (-1,3) (43) G3) 
[Can LD E) G1 
ta bd = | 3,1) Rat (1) (3,1) 





(>I 
(-3,-3)  (-1,-3) (1,-3) (3, -3) 
i a 


To calculate the probability of symbol error for M-ary QAM, we exploit the property 
that a QAM square constellation can be factored into the product of the corresponding 
PAM constellation with itself. We may thus proceed as follows: 


1. The probability of correct detection for M-ary QAM may be written as 
P, = (1 — Pty (6.58) 


where P, is the probability of symbol error for the corresponding L-ary PAM with 
L=VM. 
2. The probability of symbol error P; is defined by 


P, = (: - 4) exe $) (6.59) 


(Note that L = VM in the M-ary QAM corresponds to M in the M-ary PAM con- 
sidered in Problem 4.27.) 


3. The probability of symbol error for M-ary QAM is given by 


P =1-P, 
=1-(1- P} (6.60) 
= 2P, 


where it is assumed that P; is small enough compared to unity to justify ignoring the 
quadratic term. 


Hence, using Equations (6.58) and (6.59) in Equation (6.60), we find that the probability 
of symbol error for M-ary QAM is approximately given by 


=2l1- Eo 
Pre (1 a) ente( #2) (6.61) 


The transmitted energy in M-ary QAM is variable in that its instantaneous value 
depends on the particular symbol transmitted. It is therefore more logical to express P, in 
terms of the average value of the transmitted energy rather than Eo. Assuming that the L 
amplitude levels of the in-phase or quadrature component are equally likely, we have 


> Liz 
EE, = a|% È 2i- >| (6.62) 


i=1 
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where the multiplying factor of 2 outside the square brackets accounts for the equa] con. 
tributions made by the in-phase and quadrature components. The limits of the summation 
and the multiplying factor of 2 inside the square brackets take account of the symmetry 
nature of the pertinent amplitude levels around zero. Summing the series in Equation 
(6.62), we get 
4L - 
paai 
3 
_ 2M — 1)Ey (6.63) 
g 3 


Accordingly, we may rewrite Equation (6.61) in terms of Ea as 


1 3E ay 
P, = 21 = = ete ao (6.64) 


which is the desired result. 

The case of M = 4 is of special interest. The signal constellation for this value of M 
is the same as that for QPSK. Indeed, putting M = 4 in Equation (6.64) and noting that 
for this special case E,, equals E, where E is the energy per symbol, we find that the 
resulting formula for the probability of symbol error becomes identical to that in Equation 
(6.34), and so it should. 


QAM Cross Constellation 


To generate an M-ary QAM signal with an odd number of bits per symbol, we require 
the use of a cross constellation. As illustrated in Figure 6.18, we may construct such a 
signal constellation with bits per symbol by proceeding as follows: 


® Start with a QAM square constellation with #-1 bits per symbol. 
» Extend each side of the QAM square constellation by adding 2”~* symbols. 
> Ignore the corners in the extension. 


The inner square represents 2”-* symbols. The four side extensions add 4 x 2” ? = 27! 
symbols. The total number of symbols in the cross constellation is therefore 2”~* + 2°’, 
which equals 2” and therefore represents # bits per symbol as desired. 

Unlike QAM square constellation, it is zot possible to express a QAM cross con- 
stellation as the product of a PAM constellation with itself. The absence of such a factor- 
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FIGURE 6.18 Illustrating how a square QAM constellation can be expanded to form a QAM 
cross-constellation. 
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ization complicates the determination of the probability of symbol error P, incurred in the 
use of M-ary QAM characterized by a cross constellation. We therefore simply state the 
formula for P, without proof, as shown here 


1 [E ; 
P, = 2(1 - or ete{ E) for high Eo/No (6.65) 


which agrees with the formula of Equation (6.61) for-a square constellation, except for 
the inclusion of an extra 0.5 bit per dimension in the constellation.’ Note also that it is 
not possible to perfectly Gray code a QAM cross constellation. 


s CARRIERLESS AMPLITUDE/PHASE MODULATION 





The passband basis functions of Equations (6.53) and (6.54) assume the use of a rectan- 
gular pulse for the pulse-shaping function. For reasons that will become apparent, we 
redefine the transmitted M-ary QAM signal of Equation (6.55) in terms of a general pulse- 
shaping function g(t) as 


Ost=s=T 


k = 0, +1, +2,... 
(6.66) 


sp(t) = ayg(t — RT) cos(2af.t) — begit — kT) sin(27f.-t), 


It is assumed that carrier frequency f, has an arbitrary value with respect to the symbol 
rate 1/T. On the basis of Equation (6.66), we may express the transmitted M-ary QAM 
signal s(t) for an infinite succession of symbols as 


s(t) = X s(t) 
ie (6.67) 
= 5, [ag(t — kT) cos(2af.t) — baglt — kT) sin(27f.t)] 

kane 
This equation shows that for an arbitrary f., the passband functions g(t — kT) cos(27f.t) 
and g(t — kT) sin(27f,t) are aperiodic in that they vary from one symbol to another. 

How can we eliminate the time variations of these passband basis functions from 

symbol to symbol? To answer this question, we find it convenient to change our formalism 
from real to complex notation. Specifically, we rewrite Equation (6.67) in the equivalent 
form 


= Re Si (a, + jb,)g(t — kT) expl j2 afit) af 


(6.68) 
= oat 5 Agglt — kT) expl 2a) 
koe 
where A, is a complex number defined by 
A, = a + jb, (6.69) 


and Re{-} denotes the real part of the complex quantity enclosed inside the braces. Clearly, 
Equation (6.68) is unchanged by multiplying the summand in this equation by unity ex- 
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pressed as the product of the complex exponential exp(—j27f,kT) and its comp 


lex 
jugate exp(j27f,kT). We may thus rewrite Equation (6.68) in the new form a) 


s(t) = ref > Argit — kT) exp(j2af.t) exp(—j2af.kT) expli2mpn} 


s (6.70 
S ref D, Ag exp(j2afkT)g(t — kT) exp(j2af{t — tp} 
k=- 
Define 
Ay = Ap exp(j27f,kT) (6.71) 
&+(t) = g(t) exp(j27f.2) (6.72) 


The scalar A, is simply a rotated version of the complex representation of the coordinates 
of the kth transmitted symbol in the (,, @2)-plane. Before presenting an interpretation 
of the complex-valued signal g(t), we assume that the pulse-shaping function g(t) is 
low-pass signal whose highest frequency component is smaller than the carrier frequency 
Je- Then following the material presented in Appendix 2, we recognize g4 (t) as the analytic 
signal, or pre-envelope, representation of the band-pass signal g(t) cos(27f.t). To be more 
specific, we expand g.(t) as 


g(t) = g(t) cos(2af.t) + jg(t) sin(27f.t) 


= ple) + jt ws 
where p(t) and f(t) are defined by 
p(t) = g(t) cos(27f.t) (6.74) 
and 
P(t) = g(t) sin(2af.t) (6.75) 


We may then say that the quadrature (imaginary) component f(t) of the analytic signal 
g(t) is the Hilbert transform of the in-phase (real) component p(t). Note that whereas the 
pulse-shaping function g(t) is a baseband function, the in-phase and quadrature compo- 
nents of the corresponding analytic signal g,(t) are both passband functions. Henceforth, 
g(t) is referred to as the baseband pulse, and p(t) and f(t) are referred to as passband m- 
phase and passband quadrature pulses, respectively. 

With the definitions of A, and g(t) at hand, we are ready to finally redefine the 
transmitted signal of Equation (6.70) simply as 


s(t) = ref 5 Argi(t — en} (6.76) 
ae 


Three important observations are noteworthy from this new formulation of the transmit- 
ted signal: 


> The transmitted signal s(t) appears to be carrierless. 

> Since the formulation of s(t) in Equation (6.76) is indeed equivalent to that of the 
M-ary QAM signal presented in Equation (6.67), the new formulation of s(Z) fully 
retains the hybridized amplitude and phase modulation characterizing the origina 
M-ary QAM signal. 

> The transmitted signal s(t) represents a symbol-time-invariant realization of this by- 
brid modulation process. 


6.4 Hybrid Amplitude/Phase Modulation Schemes 375 


For a prescribed carrier frequency fe, the sequence of rotations described in Equation 
(6.71) is known. Hence, the receiver need only detect A,, in which case we may compute 
the corresponding value of A, by applying the reverse rotations as described here: 


= A, exp(—j2afkT), k= 0, +1, #2,... 
In practice, however, the rotations are ignored because they do not have any bearing on 
operation or performance of the hybrid modulation system; application of the rotations 
is in fact necessary only when its equivalence to QAM is an issue of interest. Accordingly, 
we may ignore Equation (6.71) and ‘redefine the transmitted signal of Equation (6.76) 
simply as 


s(t) = Re x Apgs(t — en} 


k=% 


£ 


Ref, 2 | (a, + jb.) p(t — RT) + jolt - vy} (6.77) 


2 [a,p(t — kT) — biĝlt — kT)] 


where, as mentioned previously, f(t) is the Hilbert transform of p(t). For obvious reasons, 
the transmitted signal of Equation (6.77) is referred to as carrierless amplitudefphase mod- 
ulation (CAP).° 

Properties of the Passband In-phase and Quadrature Pulses 


From the definitions of the passband in-phase and quadrature pulses given in Equa- 
tions (6.74) and (6.75), we deduce the following properties: 


Property 1 
The passband in-phase pulse p(t) and quadrature pulse p(t) are even and odd functions of 
time t, respectively, given that the baseband pulse g(t) is an even function of time t. 


This property follows directly from Equations (6.74) and (6.75). 


Property 2 


The passband pulses p(t) and p(t) form an orthogonal set over the entire interval (—%, œ) 
as shown by 


T p(t)p(t) dt = 0 (6.78) 


The reason for not restricting the integration interval in Equation (6.78) to a symbol period 
Tis that the pulses p(t) and f(t) are bandwidth-efficient for high-performing CAP systems. 
To prove Property 2, we transform Equation (6.78) into the frequency domain by using 
the Fourier transform to write 


f ptnte) de = fT piney) af (6.79) 


where p(t) = P(f) and pt) = = Pf). The asterisk in P*(f) denotes complex conjugation. 
(The frequency function P(f) should not be viewed as the Hilbert transform of P(f); 
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rather, following the usual terminology in Fourier analysis, P(f) is simply the Fouri 
transform of f(t).) The Fourier transform P(f) is related to the Fourier transform Ph 
(see Appendix 2) y 


Pif) = ~j sen(f)P(f) (6.80) 


where sgn(f) is the signum function. The Fourier transforms P(f) and P(f) have the same 
magnitude spectrum, but their phase spectra differ by +90 degrees for negative frequenci es 
and —90 degrees for positive frequencies. We may therefore rewrite Equation (6.79) ag 


f tet de = 5 f PAPP sent) af 
š (6.81) 
=i IPP)? sen(f) af 


Recognizing that the magnitude response |P(f)| is an even function of frequency f ang 
the signum function sgn(f) is an odd function of frequency f, the integral of Equation 
(6.81) is zero, proving Property 2. 

Next, let the passband pulses p(t) and f(z) be passed through a linear time-invariant 
channel of impulse response h(t), yielding the following passband outputs: 


u(t) = p(t) * bit) (6.82) 
and 
A(t) = p(t) x h(t) (6.83) 


where the symbol x denotes convolution. We may then formulate the third property. 


Property 3 


The passband pulses u(t) and u(t), defined in Equations (6.82) and (6.83), form a Hilbert- 
transform pair and are therefore orthogonal over the entire interval (—~, œ) for any h(t), 


It is this important property that makes it possible for the CAP receiver to separate the 
transmitted real and imaginary symbols, a, and b,, given the channel output. To prove 
Property 3, we again use the Fourier transform (in a manner similar to Equations (6.79) 
and (6.81)) to write 


[eevee ae = [oon af 
= | POH sem NPI adf 


= | APU [ALHU |? sents) df 
=0 


> EXAMPLE 6.4 Bandwidth-Efficient Spectral Shaping 


The CAP signal of Equation (6.77) uses multilevel encoding via the complex scalar As for 
spectral efficiency. We may further improve the bandwidth efficiency of CAP by using a spe 
trally efficient formulation of the baseband pulse g(t). For the selection of g(t), we may draw 
upon the raised-cosine family of pulse-shaping functions discussed in Chapter 4. 
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Normalized time, 2/T 


FIGURE 6.19 The baseband pulse g(t) for rolloff factor a = 0.2. 


Consider, for example, a baseband raised-cosine shaping filter with a rolloff factor 
a = 0.2 (i.e., excess bandwidth of 20 percent). The formula for the baseband pulse g(t) is* 


a(t) = (sineto) Sm, (6.84) 


where time ¢ is normalized with respect to the symbol duration T. Hence putting a = 0.2 in 
Equation (6.84), we get the plot of Figure 6.19. Using Equations (6.74) and (6.75), we may 
compute the passband in-phase pulse p(t) and quadrature pulse f(z) plotted in Figures 6.204 
and 6.20b for a normalized carrier frequency £,T = 0.5(1 + a) = 0.6. The waveforms of 
Figure 6.20 show that p(t) and f(t) are even and odd function of time t, respectively, in 
accordance with Property 1 and they are orthogonal over the interval (—%, œ) in accordance 
with Property 2. 
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Normalized time, ¢/T 
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FIGURE 6.20 (a) In-phase pulse p(t), and (b) quadrature pulse p(t) for rolloff factor a = 0.2. 


*Equation (6.84) is obtained by using the normalized time ¢ in place of 2 Wt in Eq. (4.62) of Chapter 4. Recall 
that the Nyquist bandwidth W is equal to 0.5 times the symbol (bit) rate. Note also that the p(t) in Chapter 4 is 
not to be confused with the p(t) in this chapter. 
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FIGURE 6.21 Block diagram of CAP transmitter. 


Basic Structure of the CAP System 


Figure 6.21 shows a functional block diagram of the CAP transmitter, which builds 
on the material embodied in Equations (6.74), (6.75), and (6.77). The transmitter Consists 
of a multilevel encoder and a pair of passband filters. The multilevel encoder partitions 
the incoming serial data stream into successive blocks of # bits each; these blocks are, in 
turn, mapped into multilevel symbols a, and bg, where k refers to the kth symbol period. 
The passband in-phase and quadrature filters process the symbol streams {a4} and {b,} in 
parallel, respectively. The impulse responses of these two filters, namely, p(t) and f(t), ate 
designed in accordance with Equations (6.74) and (6.75), for a chosen baseband pulse 
g(t). The resulting outputs of these two filters are subtracted to produce the transmitted 
CAP signal s(t) in accordance with Equation (6.77). 

The transmitted signal s(t) propagates through a channel characterized by impulse 
response b(t) and additive noise w(t). The resulting channel output is defined by 


x(t) = s(t) x b(t) + w(t) 
7 > [axp(t — RT) * b(t) — bilt — RT) x b(t] + wle) (685) 
= 5 [a,u(t — kT) — búûlt — kT)] + wit) 
i 


where u(t) and ĉ(t) are defined by Equations (6.82) and (6.83), respectively. Given x(t) as 
the input signal, the function of the receiver is to recover the transmitted symbols a, and 
b, in an optimum fashion and on a symbol-by-symbol basis. 

The receiver consists of a two-dimensional optimum receiver, the details of which: 
depend on the channel impairments. Specifically, we may mention two different situations: 


» Additive white Gaussian noise is the only impairment. In this idealized situation, the 
optimum CAP receiver consists of a two-dimensional matched filter, as indicated in 
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FIGURE 6.22 Block diagram of CAP receiver using a two-dimensional matched filter for the 
case of an ideal white Gaussian noise channel. 
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Figure 6.22. The optimum in-phase filter consists of a filter matched to the passband 
in-phase pulse p(t). The optimum quadrature filter consists of a filter matched to the 
passband quadrature pulse f(t). 

> Intersymbol interference and additive white Gaussian noise are the channel impair- 
ments. In this more realistic situation in practical terms, the structure of the optimum 
CAP receiver follows from the optimum linear receiver theory presented in Section 
4.9. In particular, the CAP receiver consists of a two-dimensional matched filter, 
followed by a pair of identical equalizers and a pair of synchronous samplers, as 
shown in Figure 6.23. The equalizers (implemented in tapped-delay-line form) com- 
pensate for dispersion in the channel. 


From Property 3, we deduce that the output of the in-phase matched filter due to the 
passband quadrature pulse is zero, and vice versa. Accordingly, in both of these situations, 
the transmitted symbols a, and b, can be detected separately by the two-dimensional 
optimum CAP receiver. 


Digital Implementation of the CAP Receiver 


The receiver structure of Figure 6.23 can be further simplified by first reformulating 
it in the form shown in Figure 6.24a, where we have introduced an analog-to-digital 
(A/D) converter at the receiver input to facilitate the use of digital signal processing. Next, 
we recognize that the matched filters and equalizers in Figure 6.244 are all linear systems. 
Accordingly, the samples at points A and B in Figure 6.24a are linear combinations of the 
input samples x(n] = x(nT,), where T; is the sampling period. It follows therefore that we 
may replace the combination of matched filter and equalizer in each receiver path in Figure 
6.24a by a single finite-duration impulse response (FIR) filter operating at the sampling 
rate 1/T,, as shown in Figure 6.246. With a common input, it is natural for the two FIR 
filters in Figure 6.246 to share a common set of unit-delay elements but different sets of 
coefficients of their own. Note also that the receiver structure of Figure 6.24 is the mirror 
image of the transmitter structure in Figure 6.21. 

One last comment is in order. In practice, the two FIR filters in the receiver of Figure 
6.24b are made adaptive so as to accommodate operation of the CAP receiver in an un- 
known environment. With adaptive equalization (filtering) as the method of choice, we 
have two options in light of the material presented in Chapter 4: linear equalization and 
decision feedback equalization (DFE). When the frequency response of the channel is ap- 
proximately flat, the use of linear equalization is adequate for the task at hand. However, 
when the frequency response of the channel and/or the noise power spectrum are not 
approximately flat, performance of the CAP receiver can be improved significantly by the 
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FIGURE 6.23 Block diagram of CAP receiver using a pair of optimum linear receivers for the 
case of a noisy, dispersive channel. 
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FIGURE 6.24 Digital implementation of the CAP receiver. (a) Reformulation of the receiver in 
Figure 6.23 using an A/D converter and associated circuitry. (b) Replacement of the matched fil- 
ter/equalizer pairs with equivalent (digitally implemented) FIR filters. 


An Application of CAP 


An important application of CAP (using 32 or 64 constellation points) is in the 
passband transmission of digital data over twisted-pair wiring of lengths less than 100 m, 
as in local area networks (LANs) suitable for premises’ distribution systems. (All modem 
LAN standards limit the length of the wiring to a maximum of 100 m; the CAP systems 
can actually operate over longer loops, if so required.) The data rates may range from 51 
up to 155 Mb/s, with the usable channel bandwidth being strictly limited to 30 MHz. 

The two major impairments for transceivers providing duplex operation over 
twisted-pairs cables are propagation loss and near-end crosstalk (NEXT); these impair- 
ments were discussed in Section 4.8, which dealt with digital subscriber lines. 

In the environment described herein, a practical issue is how to adapt the receiver to 
wide variations in twisted-pair cables. To cater to this requirement, adaptive filters are 
used to implement the optimum in-phase and quadrature filters in the receiver, as remarked 
earlier. The wider the range of twisted-pair cables that have to be accommodated, the more 
complex these adaptive filters must be. 


| 6.5 Coherent Frequency-Shift Keying 


M-ary PSK and M-ary QAM share a common property: Both are examples of linear mot 
ulation. In this section we study a nonlinear method of passband data transmission, 
namely, coherent frequency-shift keying (FSK). We begin the study by considering the 
simple case of binary FSK. 
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e Binary FSK 


In a binary FSK system, symbols 1 and 0 are distinguished from each other by transmitting 
one of two sinusoidal waves that differ in frequency by a fixed amount. A typical pair of 
sinusoidal waves is described by 


J2E, 
s(t) = T, cos(2rf;t), 0stsT, (6.86) 


0, elsewhere 


where # = 1, 2, and E, is the transmitted signal energy per bit; the transmitted frequency 
is 
neti 


f= T ` for some fixed integer z, and į = 1, 2 (6.87) 
b 





Thus symbol 1 is represented by s,(t), and symbol 0 by s2(t). The FSK signal described 
here is known as Sunde’s FSK. It is a continuous-phase signal in the sense that phase 
continuity is always maintained, including the inter-bit switching times. This form of dig- 
ital modulation is an example of continuous-phase frequency-shift keying (CPFSK), on 
which we have more to say later on in the section. 

From Equations (6.86) and (6.87), we observe directly that the signals s(t) and s2(t) 
are orthogonal, but not normalized to have unit energy. We therefore deduce that the most 
useful form for the set of orthonormal basis functions is 


dlt) = 2 cos(2rrf;t), 0stsT, (6.88) 


0, elsewhere 


where i = 1, 2. Correspondingly, the coefficient s; for i = 1, 2, andj = 1, 2 is defined by 


ell Gabe 


= f" Fe cos(2arf;t) he cos(2aft) d (6.89) 


pes x Ep, i = į 
0, i#j 
Thus, unlike coherent binary PSK, a coherent binary FSK system is characterized by having 


a signal space that is two-dimensional (i.e., N = 2) with two message points (i.e., M = 2), 
as shown in Figure 6.25. The two message points are defined by the 


sı = | A (6.90) 
0 
and 
0 
s2 = | tA (6.91) 


with the Euclidean distance between them equal to V2E,. Figure 6.25 also includes a 
couple of inserts, which show waveforms representative of signals s,(t) and s(t). 
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FIGURE 6.25 Signal-space diagram for binary FSK system. The diagram also includes two 
inserts showing example waveforms of the two modulated signals sı(t) and s2(t). 


Error Probability of Binary FSK 


The observation vector x has two elements x, and x, that are defined by, respectively, 
Tp 
x = I x(t), (t) dt (6.92) 
and 
Te 
m= f, xogan dt (6.33) 


where x(t) is the received signal, the form of which depends on which symbol was.trans- 
mitted. Given that symbol 1 was transmitted, x(t) equals s(t) + w(t), where w(t) is the 
sample function of a white Gaussian noise process of zero mean and power spectral density 
N,/2. If, on the other hand, symbol 0 was transmitted, x(t) equals s2(t) + w(t). : 
Now, applying the decision rule of Equation (5.59), we find that the observation 
space is partitioned into two decision regions, labeled Z, and Z, in Figure 6.25, The 


decision boundary, separating region Z, from region Z, is the perpendicular bisector ° 
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the line joining the two message points. The receiver decides in favor of symbol 1 if the 
received signal point represented by the observation vector x falls inside region Z4. This 
occurs when x, > x2. If, on the other hand, we have x, < x2, the received signal point 
falls inside region Z,, and the receiver decides in favor of symbol 0. On the decision 
boundary, we have x, = x2, in which case the receiver makes a random guess in favor of 
symbol 1 or 0. 

Define a new Gaussian random variable Y whose sample value y is equal to the 
difference between x, and xz; that is, i 


y = X41 — x2 (6.94) 


The mean value of the random variable Y depends on which binary symbol was trans- 
mitted. Given that symbol 1 was transmitted, the Gaussian random variables X, and X,, 
whose sample values are denoted by x, and x2, have mean values equal to VE; and zero, 
respectively. Correspondingly, the conditional mean of the random variable Y, given that 
symbol! 1 was transmitted, is 


E[Y|1] 


E[X,|1] — E[X2}1] 
+V E, 
On the other hand, given that symbol 0 was transmitted, the random variables X, and X, 
have mean values equal to zero and VE;, respectively. Correspondingly, the conditional 
mean of the random variable Y, given that symbol 0 was transmitted, is 
E[Y|0] = E[X,|0] — E[X,|0] 
= —VE, 
The variance of the random variable Y is independent of which binary symbol was trans- 


mitted, Since the random variables X, and X, are statistically independent, each with a 
variance equal to No/2, it follows that 


var[Y] 


(6.95) 


(6.96) 


Il 


var[X,] + var[X2] 
= No 


Suppose we know that symbol 0 was transmitted. The conditional probability density 
function of the random variable Y is then given by 


(6.97) 





(y + VE) | 6.58) 


fly|0) = —— exp 
27No 2No 
Since the condition x, > x2, or equivalently, y > 0, corresponds to the receiver making a 


decision in favor of symbol 1, we deduce that the conditional probability of error, given 
that symbol 0 was transmitted, is 


Pio = P(y > 0|symbol 0 was sent) 
-f fely (0) dy (6.99) 
__1 ie (y + VE x 
ViaN, Jo P 2No y 





Put 
Yy + VE, ion 


VIN, z (6.100) 
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Then, changing the variable of integration from y to z, we may rewrite Equation (6.99 
as follows: =A 


-1 Í i _,2 
Pu = = | eos exp(—z7) dz 


ot i E, (6.101) 
2 eric 2N, 


Similarly, we may show the po1, the conditional probability of error given that symbol 1 
was transmitted, has the same value as in Equation (6.101). Accordingly, averaging p 
and por, we find that the average probability of bit error or, equivalently, the bit error rate 
for coherent binary FSK is (assuming equiprobable symbols) 


-1 | Ee 
P, = 3 efe( Ze) (6.102) 


Comparing Equations (6.20) and (6.102), we see that, in a coherent binary FSK 
system, we have to double the bit energy-to-noise density ratio, E,/No, to maintain the 
same bit error rate as in a coherent binary PSK system. This result is in perfect accord with 
the signal-space diagrams of Figures 6.3 and 6.25, where we see that in a binary PSK 
system the Euclidean distance between the two message points is equal to 2V Es, whereas 
in a binary FSK system the corresponding distance is /2E,- For a prescribed E,, the 
minimum distance di, in binary PSK is therefore 1/2 times that in binary FSK. Recall 
from Chapter 5 that the probability of error decreases exponentially as din, hence the 
difference between the formulas of Equations (6.20) and (6.102). 





Generation and Detection of Coherent Binary FSK Signals 


To generate a binary FSK signal, we may use the scheme shown in Figure 6.26a. The 
incoming binary data sequence is first applied to an on-off level encoder, at the output of 
which symbol 1 is represented by a constant amplitude of VE, volts and symbol 0 is 
represented by zero volts. By using an inverter in the lower channel in Figure 6.264, we ia 
effect make sure that when we have symbol 1 at the input, the oscillator with frequency 
f, in the upper channel is switched on while the oscillator with frequency fz in the lower 
channel is switched off, with the result that frequency fı is transmitted. Conversely, when 
we have symbol 0 at the input, the oscillator in the upper channel is switched off and the 
oscillator in the lower channel is switched on, with the result that frequency fz is trans- 
mitted. The two frequencies fı and f, are chosen to equal different integer multiples of the 
bit rate 1/T,, as in Equation (6.87). 

In the transmitter of Figure 6.26a, we assume that the two oscillators are synchro- 
nized, so that their outputs satisfy the requirements of the two orthonormal basis functions 
p(t) and ġ2(ż), as in Equation (6.88). Alternatively, we may use a single keyed (voltage 
controlled) oscillator. In either case, the frequency of the modulated wave is shifted wi 
a continuous phase, in accordance with the input binary wave. , 

To detect the original binary sequence given the noisy received signal x(t), we maf 
use the receiver shown in Figure 6.265. It consists of two correlators with a common input 
which are supplied with locally generated coherent reference signals ¢,(t) and $ aft). The 
correlator outputs are then subtracted, one from the other, and the resulting difference) 
is compared with a threshold of zero volts. If y > 0, the receiver decides in favor of 1.8 
the other hand, if y < 0, it decides in favor of 0. If y is exactly zero, the receiver makes4 
random guess in favor of 1 or 0. 
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z(t) = V2, cos(2arf,t) 
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Threshold = 0 


ot) 
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FIGURE 6.26 Block diagrams for (a) binary FSK transmitter and (b) coherent binary FSK 
receiver, 


Power Spectra of Binary FSK Signals 


Consider the case of Sunde’s FSK, for which the two transmitted frequencies f, and 
fa differ by an amount equal to the bit rate 1/T,, and their arithmetic mean equals the 
nominal carrier frequency f; phase continuity is always maintained, including inter-bit 
switching times. We may express this special binary FSK signal as follows: 


dt) = el tapes |. 0=t=T, (6.103) 
Te Te 


and using a well-known trigonometric identity, we get 


= [Ee cosl + T _ [PE infa) i 

s(t) = T, cos( + ze) cos(27f.t) T, sin( 22) sin(27f.t) 
= [PEs (7 = [E inl 7) si 
= T, cos( z) cos(2mft) + T, sin( =) sin(27f.t) 


In the last line of Equation (6.104), the plus sign corresponds to transmitting symbol 0, 
and the minus sign corresponds to transmitting symbol 1. As before, we assume that the 
symbols 1 and 0 in the random binary wave at the modulator input are equally likely, and 
that the symbols transmitted in adjacent time slots are statistically independent. Then, 


(6.104) 
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based on the representation of Equation (6.104), we may make the following observatio 
pertaining to the in-phase and quadrature components of a binary FSK signal with cos 
tinuous phase: = 


1. The in-phase component is completely independent of the input binary wave, Ie 
equals V2E,/T, cos(t/T;) for all values of time t. The power spectral density of 
this component therefore consists of two delta functions, weighted by the factor 
E,/2T,, and occurring at f = +1/2T;,. 

2. The quadrature component is directly related to the input binary wave. During the 
signaling interval 0 = ¢ = T,, it equals —g(¢) when we have symbol 1, and + alt) 
when we have symbol 0. The symbol shaping function g(t) is defined by 


2E, . {wt 
— = <=tsT, 
g(t) = T, sn(2), Ost=<=T, (6.105) 


0, elsewhere 
The energy spectral density of this symbol shaping function equals 


_ 8E,T, cos*(7T,f) 
Well) = Sara — a (6.106) 


The power spectral density of the quadrature component equals V,(f)/T;,. It is also 
apparent that the in-phase and quadrature components of the binary FSK signal are in- 
dependent of each other. Accordingly, the baseband power spectral density of Sunde’s FSK 
signal equals the sum of the power spectral densities of these two components, as shown 


by 
_ Ey 1 1 8E, cos? (nT, f) 
snm) m) t anea tm 

Substituting Equation (6.107) in Equation (6.4), we find that the power spectrum of 
the binary FSK signal contains two discrete frequency components located at (f, + 1/2T,) 
= fı and (f, — 1/2T,) = fz, with their average powers adding up to one-half the total 
power of the binary FSK signal. The presence of these two discrete frequency components 
provides a means of synchronizing the receiver with the transmitter. 

Note also that the baseband power spectral density of a binary FSK signal with 
continuous phase ultimately falls off as the inverse fourth power of frequency. This is 
readily established by taking the limit in Equation (6.107) as f approaches infinity. If, 
however, the FSK signal exhibits phase discontinuity at the inter-bit switching instants 
(this arises when the two oscillators applying frequencies fı and fa operate independently 
of each other), the power spectral density ultimately falls off as the inverse square of 
frequency; see Problem 6.23. Accordingly, an FSK signal with continuous phase does not 
produce as much interference outside the signal band of interest as an FSK signal with 
discontinuous phase. 

In Figure 6.5, we have plotted the baseband power spectra of Equations (6.22) and 
(6.107). (To simplify matters, we have only plotted the results for positive frequency ‘ 
both cases, Sp(f) is shown normalized with respect to 2E,, and the frequency is normalize 
with respect to the bit rate R, = 1/T,. The difference in the falloff rates of these spect!# 
can be explained on the basis of the pulse shape g(t). The smoother the pulse, the faster 
the drop of spectral tails to zero. Thus, since binary FSK (with continuous phase) hasa 
smoother pulse shape, it has lower sidelobes than binary PSK. 
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MINIMUM SHIFT KEYING 


In the coherent detection of binary FSK signal, the phase information contained in the 
received signal is not fully exploited, other than to provide for synchronization of the 
receiver to the transmitter. We now show that by proper use of the phase when performing 
detection, it is possible to improve the noise performance of the receiver significantly. This 
improvement is, however, achieved at the expense of increased receiver complexity. 

Consider a continuous-phase frequency-shift keying (CPFSK) signal, which is defined 
for the interval 0 = t = T, as follows: 


E cos[2mfit-+ 0(0)] for symbol 1 
V Ts 


5 cos[2mf,t + @(0)] for symbol 0 
y T 


where E, is the transmitted signal energy per bit, and T; is the bit duration. The phase 
6(0), denoting the value of the phase at time ¢ = 0, sums up the past history of the 
modulation process up to time t = 0. The frequericies fı and f are sent in response to 
binary symbols 1 and 0 appearing at the modulator input, respectively. 
Another useful way of representing the CPFSK signal s(t) is to express it in the 
conventional form of an angle-modulated signal as follows: 


s(t) = jt cos[2mf.t + 0(t)] (6.109) 
b 


where @(t) is the phase of s(t). When the phase 6(t) is a continuous function of time, we 
find that the modulated signal s(t) itself is also continuous at all times, including the inter- 
bit switching times. The phase 6(t) of a CPFSK signal increases or decreases linearly with 
time during each bit duration of T, seconds, as shown by 

A(t) = @(0) + h t, 0stsT, (6.110) 

Tp, ? 

where the plus sign corresponds to sending symbol 1, and the minus sign corresponds to 
sending symbol 0; the parameter h is to be defined. Substituting Equation (6.110) into 
(6.109), and then comparing the angle of the cosine function with that of Equation (6.108), 
we deduce the following pair of relations: 


s(t) = (6.108) 


p i 

fit IT, 7 fi (6.111) 

as (6.112) 

ara: 

Solving Equations (6.111) and (6.112) for f, and h, we thus get 
1 

k=Zlht fi) (6.113) 
and 

b=Tilfi — fr) (6.114) 


The nominal carrier frequency f, is therefore the arithmetic mean of the frequencies f, and 
fi. The difference between the frequencies fı and f,, normalized with respect to the bit 
rate 1/T;,, defines the dimensionless parameter h, which is referred to as the deviation ratio. 
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Phase Trellis 
From Equation (6.110) we find that at time t = Tp, 


ah for symbol 1 
eter 0(0) pe for symbol 0 (6.115) 
That is to say, the sending of symbol 1 increases the phase of a CPFSK signal s(t) by a} 
radians, whereas the sending of symbol 0 reduces it by an equal amount. 

The variation of phase 6(t) with time ¢ follows a path consisting of a sequence of 
straight lines, the slopes of which represent frequency changes. Figure 6.27 depicts possible 
paths starting from time t = 0. A plot like that shown in Figure 6.27 is called a phase tree 
The tree makes clear the transitions of phase across interval boundaries of the incoming 
sequence of data bits. Moreover, it is evident from Figure 6.27 that the phase of a CPFSK 
signal is an odd or even multiple of wh radians at odd or even multiples of the bit duration 
T;, respectively. 

The phase tree described in Figure 6.27 is a manifestation of phase continuity, which 
is an inherent characteristic of a CPFSK signal, To appreciate the notion of phase conti- 
nuity, let us go back for a moment to Sunde’s FSK, which is a CPFSK scheme as previously 
described. In this case, the deviation ratio h is exactly unity. Hence, according to Figure 
6.27 the phase change over one bit interval is +7 radians. But, a change of +7 radians is 
exactly the same as a change of —7 radians, modulo 27. It follows therefore that in the 
case of Sunde’s FSK there is #o memory; that is, knowing which particular change occurred 
in the previous bit interval provides no help in the current bit interval. i 

In contrast, we have a completely different situation when the deviation ratio h is’ 
assigned the special value of 1/2. We now find that the phase can take on only the two 
values +7/2 at odd multiples of T,, and only the two values 0 and mat even multiples of 
Tp, as in Figure 6.28. This second graph is called a phase trellis, since a “trellis” is a treelike 
structure with remerging branches. Each path from left to right through the trellis of Figure 
6.28 corresponds to a specific binary sequence input. For example, the path shown in 
boldface in Figure 6.28 corresponds to the binary sequence 1101000 with 6(0) = 0. Hence- 
forth, we assume that h = 1/2. 




















ali) — 6(0), radians 














FIGURE 6.27 Phase tree. 
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e(t) — (0), radians 








FIGURE 6.28 Phase trellis; boldfaced path répresents the sequence 1101000. 


With = 1/2, we find from Equation (6.114) that the frequency deviation (i.e., the 
difference between the two signaling frequencies f, and f,) equals half the bit rate. This is 
the minimum frequency spacing that allows the two FSK signals representing symbols 1 
and 0, as in Equation (6.108), to be coherently orthogonal in the sense that they do not 
interfere with one another in the process of detection. It is for this reason that a CPFSK 
signal with a deviation ratio of one half is commonly referred to as minimum shift keying 


(MSK).’ 


Signal-Space Diagram of MSK 


Using a well-known trigonometric identity in Equation (6.109), we may express the 
CPFSK signal s(t) in terms of its in-phase and quadrature components as follows: 


s(t) = E cos[6(t)] cos(27f.t) — ea sin[0(t)] sin(27f,t) (6.116) 
b b ; 


Consider first the in-phase component V2E,/T, cos[6(ż)]. With the deviation ratio 
h = 1/2, we have from Equation (6.110) that 


T 
= + — <t< 

a(t) = (0) oT,” t, 0=t=T, (6.117) 
where the plus sign corresponds to symbol 1 and the minus sign corresponds to symbol 
0. A similar result holds for 6(t) in the interval —T, = t = 0, except that the algebraic 
sign is not necessarily the same in both intervals. Since the phase 6(0) is 0 or 7 depending 
on the past history of the modulation process, we find that, in the interval -T, = t = T;, 
the polarity of cos[@(t)] depends only on 0(0), regardless of the sequence a 1s and Os 
transmitted before or after t = 0. Thus, for this time interval, the in-phase component s,(t) 

consists of a half-cycle cosine pulse defined as follows: 


s(t) = Ea cos[8(ż)] 
=- [2E m. 
= T, cos[8(0)] cos s) (6.118) 
E . 
+ cos( z- ‘), -T =s t< T, 


where the plus sign corresponds to 6(0) = 0 and the minus sign corresponds to 6(0) = m. 
Ina similar way, we may show that, in the interval 0 <t = 2T,, the quadrature component 
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solt) consists of a half-cycle sine pulse, whose polarity depends only on 6(T;), as shown 


by 
pA hae 
solt) = ic sin[4(t)] 
2E, . a F T 
T sin[@(T;,)] sn( s) (6.119) 
2E; . T 
£ ieee sin( ‘), 0<t=2T, 


where the plus sign corresponds to 6(T,) = 1/2 and the minus’ sign corresponds to 
O(T,) = —7/2. 

From the foregoing discussion we see that since the phase states @(0) and 6(T;) can 
each assume one of two possible values, any one of four possibilities can arise, as described 
here: 


i 


1i 


» The phase 0(0) = 0 and 6(T;) = 7/2, corresponding to the transmission of symbol, 

> The phase 0(0) = mand 6(T;) = 7/2, corresponding to the transmission of symbol 0, 

> The phase 0(0) = wand @(T,) = —7/2 (or, equivalently, 37/2 modulo 27), cotre- 
sponding to the transmission of symbol 1. 

> The phase 0(0) = 0 and 0(T;) = —7/2, corresponding to the transmission of 
symbol 0. 


This, in turn, means that the MSK signal itself may assume any one of four possible forms, 
depending on the values of 6(0) and 6(T;). 

From the expansion of Equation (6.116), we deduce that the orthonormal basis 
functions @ ,(t) and @.(t) for MSK are defined by a pair of sinusoidally modulated quad- 
rature carriers: 


$t) = E cos r) cos(2mft), O<tST, (6.120) 
2 ./0 ; 
ob2(t) = J sin( :) sin(27f.t), O0st=T, (6.121) 
Correspondingly, we may express the MSK signal in the expanded form 
s(t) = sıġ (t) + s2@2(t), 0<t=<=T, (6.122) 


where the coefficients sı and sz are related to the phase states 0(0) and 6(T;), respectively. 
To evaluate s,, we integrate the product s(t) (t) between the limits — Ta and T,, as shown 
by 
Tp 
s = Í s(t) (t) dt 
-T 
= VE, cos[6(0)], -Te =t =T, 


Similarly, to evaluate sz we integrate the product s(t) 2(t) between the limits 0 and Tas 
as shown by 


(6.123) 


2T, 
s= Í s(t)ba(t) dt (6.124 
= —VE, sin[6(T;)I, O=st<2T, 
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FIGURE 6.29 Signal-space diagram for MSK system. 


7 


Note that in Equations (6.123) and (6.124): 


> Both integrals are evaluated for a time interval equal to twice the bit duration. 

> Both the lower and upper limits of the product integration used to evaluate the 
coefficient s, are shifted by the bit duration T, with respect to those used to evaluate 
the coefficient s2. 

» The time interval 0 = ¢ = T,, for which the phase states 6(0) and 8(T,) are defined, 
is common to both integrals. 


Accordingly, the signal constellation for an MSK signal is two-dimensional (i.e., 
N = 2), with four possible message points (i.e., M = 4), as illustrated in Figure 6.29. 
The coordinates of the message points are as follows in a counterclockwise direction: 
(+VEs, +VEz), (VEs +VEz)s (VEs —VE,), and (+VE,, —VE;). The possible 
values of 6(0) and @(T;,), corresponding to these four message points, are also included in 
Figure 6.29. The signal-space diagram of MSK is thus similar to that of QPSK in that both 
of them have four message points. However, they differ in a subtle way that should be 
carefully noted: In QPSK the transmitted symbol is represented by any one of the four 
message points, whereas in MSK one of two message points is used to represent the trans- 
mitted symbol at any one time, depending on the value of 9(0). 

Table 6.5 presents a summary of the values of 6(0) and 6(T;,), as well as the corre- 
sponding values of sı and s; that are calculated for the time intervals -T, = ¢ = T, and 
0 = ż = 2T,, respectively. The first column of this table indicates whether symbol 1 or 
symbol 0 was sent in the interval 0 = ¢ = T,. Note that the coordinates of the message 
points, sı and s2, have opposite signs when symbol 1 is sent in this interval, but the same 
sign when symbol 0 is sent. Accordingly, for a given input data sequence, we may use the 
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TABLE 6.5 Signal-space 
characterization of MSK 





Coordinates 
Phase States of Message 
Transmitted (radians) Points 
Binary Symbol, — — 
o0=ts=T, 6(0) 6(T,) S; S2 
0 0 —al2 +VE, +VE, 
1 T —nl2 =V E; +V E; 
0 T +a/2 —VE, —VE, 
1 0 +2 +VE, -VE 





entries of Table 6.5 to derive, on a bit-by-bit basis, the two sequences of coefficients re- 
quired to scale (t) and #2(t), and thereby determine the MSK signal s(t). 


b> EXAMPLE 6.5 


Figure 6.30 shows the sequences and waveforms involved in the generation of an MSK signal 
for the binary sequence 1101000. The input binary sequence is shown in Figure 6.30a. The 
two modulation frequences are: fı = 5/4T, and f, = 3/4T,. Assuming that, at time t = 0 the 
phase 6(0) is zero, the sequence of phase states is as shown in Figure 6.30, modulo 27, The 
polarities of the two sequences of factors used to scale the time functions $,(t) and z(t) are 
shown in the top lines of Figures 6.30b and 6.30c. Note that these two sequences are offset 
relative to each other by an interval equal to the bit duration T,. The waveforms of the 
resulting two components of s(£), namely, s161(t) and s2$2(¢), are also shown in Figures 6.306 
and 6.30c. Adding these two modulated waveforms, we get the desired MSK signal s(t) shown 
in Figure 6.30d. 


Error Probability of MSK 
In the case of an AWGN channel, the received signal is given by | 
x(t) = s(t) + w(t) 


where s(t) is the transmitted MSK signal, and w(t) is the sample function of a white Gaus- 
sian noise process of zero mean and power spectral density Ng/2. To decide whether 
symbol 1 or symbol 0 was transmitted in the interval 0 = t = T,, say, we have to establish 
a procedure for the use of x(t) to detect the phase states 6(0) and 6(T,). For the optimum 
detection of @(0), we first determine the projection of the received signal x(t) onto the 
reference signal @,(t) over the interval —T, = t = Ta, obtaining 


Ts 
x = i x(t) (t) dt 


=s,+w, -T,= t< T, 


(6.125) 


where s, is as defined by Equation (6.123) and w is the sample value of a Gaussian random 
variable of zero mean and variance No/2. From the signal-space diagram of Figure 6.295 
we observe that if x, > 0, the receiver chooses the estimate 6(0) = 0. On the other hand, 
if x, < 0, it chooses the estimate 6(0) = 7. 
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FIGURE 6.30 (a) Input binary sequence. (b) Waveform of scaled time function sii (t). 
(c) Waveform of scaled time function s.3(t). (d) Waveform of the MSK signal s(t) obtained by 
adding s1% (t) and s(t) on a bit-by-bit basis. 


Similarly, for the optimum detection of 6(T,), we determine the projection of the 
received signal x(t) onto the second reference signal (t) over the interval 0 = t = 2Tp, 
obtaining 


OT, 
x2 = i x(t) (t) dt 


= s, + uw, 0st 2T, 


(6.126) 


where s; is as defined by Equation (6.124) and w is the sample value of another indepen- 
dent Gaussian random variable of zero mean and variance No/2. Referring again to the 
signal space diagram of Figure 6.29, we observe that if x. > 0, the receiver chooses the 
estimate 6(T;) = —7/2. If, on the other hand, x2 < 0, it chooses the estimate 6(T,) = 7/2. 

To reconstruct the original binary sequence, we interleave the above two sets of phase 
decisions, as described next (see Table 6.5): 


œ If we have the estimates 6(0) = 0 and &(T,) = —7/2, or alternatively if we have the 
estimates (0) = a and 6(T,) = 7/2, the receiver makes a decision in favor of sym- 
bol 0. 
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> If we have the estimates (0) = mand 6(T;) = —7/2, or alternatively if we have th 
estimates 6(0) = 0 and 6(T,) = 77/2, the receiver makes a decision in favor of an 
bol 1. ` 


Thus referring to the signal-space diagram of Figure 6.29, we see that the decision 
made by the receiver is between the message points m and ms for symbol 1, or between 
the message points m and mz, for symbol 0. The corresponding decisions whether 6(0) ig 
0 or m and whether 6(T;,) is — 7/2 or +7/2 (i.e., the bit decisions) are made alternately in 
the I- and Q-channels of the receiver, with each channel looking at the input signal fo, 
2T, seconds. The signal from other bits does not interfere with the receiver’s decision for 
a given bit in either channel. The receiver makes an error when the I-channel assigns the 
wrong value to 6(0) or the Q-channel assigns the wrong value to 6(T;,). Accordingly, Using 
the statistical characterizations of the product-integrator outputs x1 and x of these two 
channels, defined by Equations (6.125) and (6.126), respectively, we readily find that the 
bit error rate for coherent MSK is given by 


ae [Es 
P, = 5 ete{ Es) (6.127) 


which is exactly the same as that for binary PSK and QPSK. It is important to note, 
however, that this good performance is the result of the detection of the MSK signal being 
performed in the receiver on the basis of observations over 2T, seconds. 


Generation and Detection of MSK Signals 


Consider next the generation and demodulation of MSK. Figure 6.314 shows the 
block diagram of a typical MSK transmitter. The advantage of this method of generating 
MSK signals is that the signal coherence and deviation ratio are largely unaffected by 
variations in the input data rate. Two input sinusoidal waves, one of frequency fe = n/4T, 
for some fixed integer no and the other of frequency 1/4T,, are first applied to a product 
modulator. This produces two phase-coherent sinusoidal waves at frequencies f, and fy, 
which are related to the carrier frequency f; and the bit rate 1/T, by Equations (6.111) 
and (6.112) for h = 1/2. These two sinusoidal waves are separated from each other by 
two narrowband filters, one centered at f, and the other at fz. The resulting filter outputs 
are next linearly combined to produce the pair of quadrature carriers or orthonormal basis 
functions ¢ ,(¢) and 2(t). Finally, 61(¢) and #,(£) are multiplied with two binary waves 
a(t) and a(t), both of which have a bit rate equal to 1/2T,. These two binary waves are 
extracted from the incoming binary sequence in the manner described in Example 6.5. 

Figure 6.31 shows the block diagram of a typical MSK receiver. The received signal 
x(t) is correlated with locally generated replicas of the coherent reference signals @ ,(¢) and 
2(t). Note that in both cases the integration interval is 2T, seconds, and that the inte- 
gration in the quadrature channel is delayed by T, seconds with respect to that in the in- 
phase channel. The resulting in-phase and quadrature channel correlator outputs, *1 and 
xz, are each compared with a threshold of zero, and estimates of the phase 6(0} and (Ti) 
are derived in the manner described previously. Finally, these phase decisions are intet- 
leaved so as to reconstruct the original input binary sequence with the minimum average 
probability of symbol error in an AWGN channel. 


Power Spectra of MSK Signals 


As with the binary FSK signal, we assume that the input binary wave is random, 
with symbols 1 and 0 equally likely, and the symbols transmitted during different tim 
slots being statistically independent. In this case, we make the following observations: 
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FIGURE 6.31 Block diagrams for (a) MSK transmitter and (b) coherent MSK receiver. 


1. Depending on the value of phase state 6(0), the in-phase component equals +g(t) or 
~g(t), where 





2E, Tt 
gt) =7y T, cos( 7"), heeds (6.128) 


; otherwise 


The energy spectral density of this symbol-shaping function is 


_ 32E,T, | cos(2aT,f) T? 
hlf) = Sar | ee = | (6.129) 


Hence, the power spectral density of the in-phase component equals ,(f)/2T». 
2. Depending on the value of the phase state 6(T;), the quadrature component equals 
+g(t) or —g(t), where we now have 


2E, . Tt 
gt) =i yT, s(a) Etsai (6.130) 


otherwise 
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The energy spectral density of this second symbol-shaping function is also given b 
Equation (6.129). Hence, the in-phase and quadrature components have the sam 
power spectral density. . 
3. The in-phase and quadrature component of the MSK signal are also statistica] 
independent. Hence, the baseband power spectral density of the MSK signal is sive, 








by 
Salf) = 2| eel | 
32E, | cos(2aTaf) T (6.131) 
wm | 1672? - 1 


The baseband power spectrum of Equation (6.131) is plotted in Figure 6.9, where 
the power spectrum is normalized with respect to 4E, and the frequency f is normalized 
with respect to the bit rate 1/T,,. This figure also includes the corresponding plot of Equa. 
tion (6.40) for the QPSK signal, which was considered earlier. For f >> 1/T,, the baseband 
power spectral density of the MSK signal falls off as the inverse fourth power of frequency, 
whereas in the case of the QPSK signal it falls off as the inverse square of frequency, 
Accordingly, MSK does not produce as much interference outside the signal band of in. 
terest as QPSK. This is a desirable characteristic of MSK, especially when the digital com- 
munication system operates with a bandwidth limitation. 


© GAUSSIAN-FILTERED MSK 


From the detailed study of minimum shift keying (MSK) just presented, we may summarize 
the desirable properties of the MSK signal as follows: 


» Constant envelope 
» Relatively narrow bandwidth 
> Coherent detection performance equivalent to that of QPSK 


However, the out-of-band spectral characteristics of MSK signals, as good as they are, still 
do not satisfy the stringent requirements of certain applications such as wireless commu 
nications, To illustrate this limitation, we find from Equation (6.131) that at fT, = 0.5, 
the baseband power spectral density of the MSK signal drops by only 10 logiy 9 = 9.54 
dB below its midband value. Hence, when the MSK signal is assigned a transmission 
bandwidth of 1/T;, the adjacent channel interference of a wireless communication system 
using MSK is not low enough to satisfy the practical requirements of such a multiuser 
communications environment, 

Recognizing that the MSK signal can be generated by direct frequency modulation 
of a voltage-controlled oscillator, we may overcome this serious limitation of MSK by 
modifying its power spectrum into a compact form, while maintaining the constant 
envelope property of the MSK signal. This modification can be achieved through the use 
of a premodulation low-pass filter, hereafter referred to as a baseband pulse-shaping filter. 
Desirably, the pulse-shaping filter should satisfy the following properties: 


1, Frequency response with narrow bandwidth and sharp cutoff characteristics. 
2. Impulse response with relatively low overshoot. 
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3. Evolution of a phase trellis where the'carrier phase of the modulated signal assumes 
the two values +7/2 at odd multiples of T, and the two values 0 and ~ at even 
multiples of T, as in MSK. 


Condition 1 is needed to suppress the high-frequency components of the transmitted signal. 
Condition 2 avoids excessive deviations in the instantaneous frequency of the FM signal. 
Finally, condition 3 ensures that the modified FM signal can be coherently detected in the 
same way as the MSK signal, or it can be noncoherently detected as a simple binary FSK 
signal. 

These desirable properties can be achieved by passing a nonreturn-to-zero (NRZ) 
binary data stream through a baseband pulse-shaping filter whose impulse response (and 
likewise its frequency response) is defined by a Ganssian function. The resulting method 
of binary frequency modulation is naturally referred to as Gaussian-filtered MSK or just 
GMSK.® 

Let W denote the 3 dB baseband bandwidth of the pulse-shaping filter. We may then 
define the transfer function H(f) and impulse response h(t) of the pulse-shaping filter as 


follows, respectively: 
l 2 
Hif) = op( -22 (£) ) (6.132) 


and 
<T Wexp joe W?2? (6.133) 
log2 i 


The response of this Gaussian filter to a rectangular pulse of unit amplitude and 
duration T; (centered on the origin) is given by (see Problem 6.28) 


Ty/2 
j=f h(t — t) dr 


T,/2 


ln Ty/2 (6.134) 
eo, 2it — 
=y log2 wf Ty/2 en = Ww Eae 2) ar 


which may be expressed as the difference between two complementary error functions, as 
shown by 


gi) = Tan w(t = 3) = erte( nics wrt + ))| (6.135) 


The pulse response g(t) constitutes the frequency shaping pulse of the GMSK modulator, 
with the dimensionless time-bandwidth product WT, playing the role of a design 
parameter, 

The frequency-shaping pulse g(t), as defined in Equation (6.135), is noncausal in that 
it is nonzero for t < —T,/2, where t = ~T;/2 is the time at which the input rectangular 
pulse (symmetrically. positioned around the origin) is applied to the Gaussian filter. For a 
causal response, g(t) must be truncated and shifted in time. Figure 6.32 presents plots of 
g(t), which has been truncated at £ = +2.5T, and then shifted in time by 2.5T,. The plots 
shown here are for WT; = 0.2, 0.25, and 0.3. Note that as WT; is reduced, the time spread 
of the frequency-shaping pulse is correspondingly increased. 
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FIGURE 6.32 Frequency-shaping pulse g(t) of Equation (6.135) shifted in time by 2.5T, and 
truncated at +2.5T, for varying time-bandwidth product WT. 


Figure 6.33 shows the machine-computed power spectra of MSK signals (expressed 
in decibels) versus the normalized frequency difference (f — f£)T;, where f, is the mid- 
band frequency and Te is the bit duration.? The results plotted in Figure 6.33 are for 
varying values of the time-bandwidth product WT,. From this figure we may make the 
following observations: 


> The curve for the limiting condition WT, = © corresponds to the case of ordinary 
MSK. 

> When WT; is less than unity, increasingly more of the transmit power is concentrated 
inside the passband of the GMSK signal. 


An undesirable feature of GMSK is that the processing of NRZ binary data bya 
Gaussian filter generates a modulating signal that is no longer confined to a single bit 
interval as in ordinary MSK, which is readily apparent from Figure 6.32. Stated in another 
way, the tails of the Gaussian impulse response of the pulse-shaping filter cause the mod- 
ulating signal to spread out to adjacent symbol intervals. The net result is the generation 
of intersymbol interference, the extent of which increases with decreasing WT,. In light 
of this observation and the observation we made on the basis of Figure 6.33 on the power 
spectra of GMSK signals, we may say that the choice of the time-bandwidth product WT, 
offers a trade-off between spectral compactness and performance loss. 

To explore the issue of performance degradation, consider the probability of error 
P, of GMSK using coherent detection in the presence of additive white Gaussian noise. 
Recognizing that GMSK is a special kind of binary frequency modulation, we may express 
P, by the empirical formula 





1 aE, 

P.=5 6.136) 
.=3 ete{ 3 E) ( 
where, as before, E, is the signal energy per bit and No/2 is the noise spectral density. The 


factor æ is a constant whose value depends on the time-bandwidth product WT). Com 
paring the formula of Equation (6.136) for GMSK with that of Equation (6.127) for 
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FIGURE 6.33 Power spectra of MSK and GMSK signals for varying time-bandwidth product. 
(Reproduced with permission from Dr. Gordon Stiiber, Georgia Tech.) 


ordinary MSK, we may view 10log;(a/2), expressed in decibels, as a measure of perfor- 
mance degradation of GMSK (with prescribed WT,) compared to ordinary MSK. Figure 
6.34 shows the machine-computed value of 10logio(a/2) versus WT). For ordinary MSK 
we have WT, = œ, in which case Equation (6.136) with a = 2 assumes exactly the same 
form as Equation (6.127) and there is no degradation in performance, which is confirmed 
by Figure 6.34. For GMSK with WT, = 0.3 we find from Figure 6.34 that there is a 





Degradation (dB) 











| 
0 0.2 0.4 0.6 0.8 
Time-bandwidth product, WT, 





FIGURE 6.34 Theoretical E,/No degradation of GMSK for varying time-bandwidth product. 
(Taken from Murata and Hirade, 1981, with permission of the IEEE.) 
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degradation in performance of about 0.46 dB, which corresponds to (a/2) = 0.9, This 
degradation in performance is a small price to pay for the highly desirable spectral com, 
pactness of the GMSK signal. 


P EXAMPLE 6.6 GMSK for GSM Wireless Communications 


An important application of GMSK is in a standardized wireless communication system widel, 

known as GSM, which is a time-division multiple-access system that is discussed in Chase 
8. For this application, the time-bandwidth product WT, of GMSK is standardized at 0,3 

which provides the best compromise between increased bandwidth occupancy and resistance 
to co-channel interference. Ninety-nine percent of the radio frequency (RF) power of GMSK 
signals so specified is confined to a bandwidth of 250 kHz, which means that, for all practical 
purposes, the sidelobes are virtually zero outside this frequency band. 

The available spectrum is divided into 200 kHz-wide subchannels. Each subchannel is 
assigned to a GSM system transmitting data at 271 kb/s. Figure 6.35 depicts the power spec. 
trum of a subchannel in relation to its two adjacent subchannels; this plot is the passband 
version of the baseband power spectrum of Figure 6.33 corresponding to WT, = 0.3. From 
Figure 6.35 we may make the following important observation: The RF power spectrum of 
the subchannel shown shaded is down by an amount larger than 40 dB at the carrier fre. 
quencies of both adjacent subchannels, which means that the effect of co-channel interference 
is practically negligible. <4 


u M-ary FSK 
Consider next the M-ary version of FSK, for which the transmitted signals.are defined by 


s(t) = E cos| Z (n. + a|, O<t<T (6.137) 
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FIGURE 6.35 Power spectrum of GMSK signal for GSM wireless communications. 
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where i = 1, 2,..., M, and the carrier frequency f. = n,/2T for some fixed integer n. 
The transmitted symbols are of equal duration T and have equal energy E. Since the 
individual signal frequencies are separated by 1/2T Hz, the signals in Equation (6.137) are 
orthogonal; that is 


T 
| si(t)s(t) dt = 0, i#*j (6.138) 

This property of M-ary FSK suggests that we may use the transmitted signals s,(t) 
themselves, except for energy normalization, as a complete orthonormal set of basis func- 
tions, as shown by 


1 O0OstsT e446 
¢;(t) VE itt i=1,2,...,M (6.139) 
Accordingly, the M-ary FSK is described by an M-dimensional signal-space diagram. 

For coherent M-ary FSK, the optimum receiver consists of a bank of M correlators 
or matched filters, with the ¢;(t) of Equation (6.139) providing the pertinent reference 
signals. At the sampling times t = kT, the receiver makes decisions based on the largest 
matched filter output in accordance with the maximum likelihood decoding rule. An exact 
formula for the probability of symbol error is however difficult to derive for a coherent 
M-ary FSK system. Nevertheless, we may use the union bound of Equation (5.96) of 
Chapter 5 to place an upper bound on the average probability of symbol error for M-ary 
FSK. Specifically, noting that the minimum distance dmin in M-ary FSK is V2E, the use of 
Equation (5.96) yields (assuming equiprobable symbols) 


1 fE\ . 
P: = 2 (M w 1) erte( EŻ) (6.140) 


For fixed M, this bound becomes increasingly tight as E/No is increased. Indeed, it becomes 
a good approximation to P, for values of P, = 10°. Moreover, for M = 2 (i.e., binary 
FSK), the bound of Equation (6.140) becomes an equality. 


Power Spectra of M-ary FSK Signals 


The spectral analysis of M-ary FSK signals’? is much more complicated than that of 
M-ary PSK signals. A case of particular interest occurs when the frequencies assigned to 
the multilevels make the frequency spacing uniform and the frequency deviation k = 0.5. 
That is, the M signal frequencies are separated by 1/2T, where T is the symbol duration. 
For k = 0.5, the baseband power spectral density of M-ary FSK signals is plotted in Figure 
6.36 for M = 2, 4, 8. 


Bandwidth Efficiency of M-ary FSK Signals 


When the orthogonal signals of an M-ary FSK signal are detected coherently, the 
adjacent signals need only be separated from each other by a frequency difference 1/2T so 
as to maintain orthogonality. Hence, we may define the channel bandwidth required to 
transmit M-ary FSK signals as 


B=— (6.141) 
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FIGURE 6.36 Power spectra of M-ary PSK signals for M = 2, 4, 8. 


For multilevels with frequency assignments that make the frequency spacing uniform and 
equal to 1/2T, the bandwidth B of Equation (6.141) contains a large fraction of the signal 
power. This is readily confirmed by looking at the baseband power spectral plots shown 
in Figure 6.36. From Equation (6.48) we recall that the symbol period T is equal to 
T, log, M. Hence, using R, = 1/T;, we may redefine the channel bandwidth B for M-ary 
FSK signals as 


RM 
= 2 log, M (6.142) 
The bandwidth efficiency of M-ary signals is therefore 
p= Bs 
R (6.143) 
_2 log, M 
M 


Table 6.6 gives the values of p calculated from Equation (6.143) for varying M. 

Comparing Tables 6.4 and 6.6, we see that increasing the number of levels M tends 
to increase the bandwidth efficiency of M-ary PSK signals, but it also tends to decrease the 
bandwidth efficiency of M-ary FSK signals. In other words, M-ary PSK signals are spec- 
trally efficient, whereas M-ary FSK signals are spectrally inefficient. 


TABLE 6.6 Bandwidth efficiency of M-ary 
FSK signals 


M 2 4 8 16 32 64 





p (bits/s/Hz) 1 1 0.75 0.5 0.3125 0.1875 





6.6 Detection of Signals with Unknown Phase 403 


6.6 Detection of Signals with 
Unknown Phase 


Up to this point in our discussion, we have assumed that the receiver is perfectly synchro- 
nized to the transmitter, and the only channel impairment is noise. In practice, however, 
it is often found that in addition to the uncertainty due to channel noise, there is also 
uncertainty due to the randomness of certain signal parameters. The usual cause of this 
uncertainty is distortion in the transmission medium. Perhaps the most common random 
signal parameter is the carrier phase, which is especially true for narrowband signals. For 
example, transmission over a multiplicity of paths of different and variable lengths, or 
rapidly varying delays in the propagating medium from transmitter to receiver, may cause 
the phase of the received signal to change in a way that the receiver cannot follow. Syn- 
chronization with the phase of the transmitted carrier may then be too costly, and the 
designer may simply choose to disregard the phase information in the received signal at 
the expense of some degradation in noise performance. A digital communication receiver 
with no provision made for carrier phase recovery is said to be noncoberent. 


@ OPTIMUM QUADRATIC RECEIVER 
Consider a binary digital communication system in which the transmitted signal is 


RE 0stsT 
s(t) = T cos(2mfit), j=12 (6.144) 


where E is the signal energy, T is the duration of the signaling interval, and the carrier 
frequency f; for symbol 7 is an integral multiple of 1/2T. The system is assumed to be 
noncoherent, in which case the received signal for an AWGN channel may be written in 
the forn 


= = 
x(t) = Z Onir i u DITT (6.145) 
T i= 1,2, 
where 9 is the unknown carrier phase, and w(t) is the sample function of a white Gaussian 
noise process of zero mean and power spectral density No/2. In a real-life situation it is 
realistic to assume complete lack of prior information about @ and to treat it as a random 
variable with uniform distribution: 


1 
flea WR Tees (6.146) 
0, otherwise 


The binary detection problem to be solved may now be stated as follows: 


Given the received signal x(t) and confronted with the unknown carrier phase 6, 
design an optimum receiver for detecting symbol s; represented by the signal com- 
ponent VE/2T cos(2mft + 0) that is contained in x(t). 


Proceeding in a manner similar to that described in Sections 5.3-5.6, we may formulate 
the conditional likelihood function of symbol s; given the carrier phase 6, as 


T 
L(s;(0)) = ox( a), x(t) cos(2afjt + 0) i) (6.147) 
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To proceed further, we have to remove dependence of L(s;(@)) on phase 8, which is achieved 
by integrating it over all possible values of 6. We may thus write 


L{s) = E L(s;(6))fa(6) de 


i T {6.148 
- = x cx es Í x(t) cos(2aft + 0) a) da ! 


Note that the dependence on @ was removed by integrating the likelihood function anq 
not the log-likelihood function. 
Using a well-known trigonometric formula, we may expand cos(2 ft + 6) as 


cos(2aft + 8) = cos(27f;t) cos 6 — sin(27f;t) sin 8 


Correspondingly, we may rewrite the integral in the exponent of Equation (6.148) as 
T T T 
[ x(t) cos(2afit + 0) dt = cos of x(t) cos(2rf;t) dt — sin of x(t) sin(2aft) dé (6.149) 


Define 
1/2 


z 2 
+ (f x(t) sin(2rf;t) ar) | (6.150) 


2 


T 
L= IU; x(t) cos(27f;t) a) 


% 
Í x(t) sin(2mf;t) dt 
1| Jo 


B: = tan™! | — (6.151) 
N x(t) cos(2af;t) dt 
Hence, we may go one step further and simplify Equation (6.149) to 
T 
f x(t) cos(2mft + 8) dt = 1,(cos 0 cos B; — sin 0 sin B;) 
p (6.152) 


= |, cos(@+ ß;) 


_ Accordingly, using Equation (6.152) in Equation (6.148), we obtain 


1” E 
Lís) = on l-a ao ep cos(@ + pa) de 
1 a+ B; E 
T 2m Nor! 6.153 
2a J-r+B; a L; cos o) dé ( ) 
1 7 E 
Imd- o Es I, cos o) do 


From Appendix 3 on Bessel functions, we immediately recognize the integral of Equa- 
tion (6.153) as the modified Bessel function of zero order: 


JE 1 E 
— |. | = — Bise A 6.154) 
i NT 1) Zz- exo Nor l; cos o) do { 
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Hence, we may express the likelihood function for the signal detection problem described 


herein in the compact form 
E 
Lis) = 1 er 1) (6.155) 


The binary hypothesis test (i.e., the hypothesis that signal s,(z) or signal s2(t) was 
transmitted) can now be written as 


E Hı E 
ages n) = age n) (6:438) 


where hypothesis H, and H correspond to signals s,(¢) and s2(t), respectively. From Ap- 
pendix 3 we note that the modified Bessel function I(-) is a monotonically increasing func- 
tion of its argument. Hence the hypothesis test can be carried out in terms of either 
I,(VE/NoTI,) or simply l;. For convenience of implementation, however, the hypothesis 
test is carried out in terms of Ê instead of };, as shown by 


BZB (6.157) 


A receiver based on Equation (6.157) is known as the quadratic receiver. In light of the 
definition of }; given in Equation (6.150), the receiver structure for computing l; is as shown 
in Figure 6.374. Note that the test described in Equation (6.157) is independent of the 
symbol energy E. It is for this reason that this hypothesis test is said to be uniformly most 
powerful with respect to the symbol energy E. 


& Two EQUIVALENT FORMS OF THE QUADRATIC RECEIVER 


We next derive two equivalent forms of the quadrature receiver shown in Figure 6.37a. 
The first form is obtained easily by replacing each correlator in Figure 6.374 with a cor- 
responding equivalent matched filter. We thus obtain the alternative form of quadrature 
receiver shown in Figure 6.376. In one branch of this receiver, we have a filter matched 
to the signal cos(2 aft), and in the other branch we have a filter matched to sin(2 ft), 
both of which are defined for the time interval 0 = ¢ = T. The filter outputs are sampled 
at time t = T, squared, and then added together. 

To obtain the second equivalent form of the quadrature receiver, suppose we have 
a filter that is matched to s(t) = cos(2mft + 8) for 0 = ¢ = T. The envelope of the matched 
filter output is obviously unaffected by the value of phase 8. Therefore, for convenience, we 
may simply choose a matched filter with impulse response cos[27f;(T — t)], corresponding 
to 6 = 0. The output of such a filter in response to the received signal x(t) is given by 


T 
y(t) = f x(7) cos[2rf(T — t + 7)] dr 
o ‘ F (6.158) 
= cos|2rf(T — t)] Í x(T) cos(2afr) dr — sin[2af(T — t)] Í x(7) sin(2afir) dr 


The envelope of the matched filter output is proportional to the square root of the sum of 
the squares of the integrals in Equation (6.158). The envelope, evaluated at time £ = T, is 


therefore 
T 2 T 24 V2 
l= {If x(t) cos(2 afr) ar] + i x(t) sin(2 afr) ar] } (6.159) 
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FIGURE 6.37 Noncoherent receivers. (a) Quadrature receiver using correlators. (b) Quadrature 
receiver using matched filters. (c) Noncoherent matched filter. 


But this is just the output of the quadrature receiver. Therefore, the output (at time T) of 
a filter matched to the signal cos(2aft + 6), of arbitrary phase 0, followed by an envelope 
detector is the same as the corresponding output of the quadrature receiver of Figure 6.374. 
This form of receiver is shown in Figure 6.37c. The combination of matched filter and 
envelope detector shown in Figure 6.37c is called a noncoberent matched filter. 

The need for an envelope detector following the matched filter in Figure 6.37¢ may 
also be justified intuitively as follows. The output of a filter matched to a rectangular RF 
wave reaches a positive peak at the sampling instant t = T. If, however, the phase of the 
filter is not matched to that of the signal, the peak may occur at a time different from the 
sampling instant. In actual fact, if the phases differ by 180 degrees, we get a negative pea 
at the sampling instant. Figure 6.38 illustrates the matched filter output for the two limiting 
conditions: @ = 0 and @ = 180 degrees. To avoid poor sampling that arises in the absence 
of prior information about the phase @, it is reasonable to retain only the envelope of the 
matched filter output, since it is completely independent of the phase mismatch 8. 
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FIGURE 6.38 Output of matched filter for a rectangular RF wave: (a) 6 = 0, and (b) 6 = 180 
degrees. 


| 6.7 Noncoherent Orthogonal Modulation 


With the noncoherent receiver structures of Figure 6.37 at our disposal, we may now 
proceed to study the noise performance of noncoherent orthogonal modulation that in- 
cludes two noncoherent receivers as special cases: noncoherent binary frequency-shift key- 
ing and differential phase-shift keying. 

Consider a binary signaling scheme that involves the use of two orthogonal signals 
$1(t) and s(t), which have equal energy. During the interval 0 = t < T, one of these two 
signals is sent over an imperfect channel that shifts the carrier phase by an unknown 
amount. Let g,(¢) and g(t) denote the phase-shifted versions of s,(t) and S(t), respectively. 
It is assumed that the signals g,(t) and g,(t) remain orthogonal and have the same energy 
E, regardless of the unknown carrier phase. We refer to such a signaling scheme as non- 
coherent orthogonal modulation. Depending on how we define the orthogonal pair of 
signals s,(t) and s(t), noncoherent binary FSK and DPSK may be treated as special cases 
of this modulation scheme. 

The channel also introduces an additive white Gaussian noise w(t) of zero mean and 


power spectral density Nj/2. We may thus express the received signal x(t) as 
p: 
9 = g(t) + w(t), sı(t) sent, O< t=T (6.160) 
g(t) + w(t), salt) sent, 0 = ż =T ` 


The requirement is to use x(t) to discriminate between $,(t) and s2(t), regardless of the 
carrier phase. 
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For this purpose, we employ the receiver shown in Figure 6.39a. The receiver consig, 
of a pair of filters matched to the transmitted signals s,(¢) and s,(t). Because the carte, 
phase is unknown, the receiver relies on amplitude as the only possible discriminant, Ao 
cordingly, the matched filter outputs are envelope detected, sampled, and then compared 
with each other. If the upper path in Figure 6.39a has an output amplitude /, greater than 
the output amplitude l, of the lower path, the receiver makes a decision in favor of s (8) 
If the converse is true, it decides in favor of s2(t). When they are equal, the decision may 
be made by flipping a fair coin. In any event, a decision error occurs when the matched 
filter that rejects the signal component of the received signal x(ż) has a larger output am. 
plitude (due to noise alone) than the matched filter that passes it. 

From the discussion presented in Section 6.6, we note that a noncoherent matched 
filter (constituting the upper or lower path in the receiver of Figure 6.39a) may be viewed 
as being equivalent to a quadrature receiver. The quadrature receiver itself has two chan. 
nels. One version of the quadrature receiver is shown in Figure 6.396. In the upper channel, 
called the in-phase channel, the received signal x(t) is correlated with the function y(t), 
which represents a scaled version of the transmitted signal s(t) or s(é) with zero carrier 
phase. In the lower channel, called the quadrature channel, on the other hand, x(t) is 
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FIGURE 6.39 (a) Generalized binary receiver for noncoherent orthogonal modulation. (b) Quad- 
rature receiver equivalent to either one of the two matched filters in part (a); the index i = 1,2. 
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correlated with another function (t), which represents the version of y(t) that results 
from shifting the carrier phase by —90 degrees. Naturally, y;(t) and ĝ;(t) are orthogonal 
to each other. 

The signal ĝ;(t) is in fact the Hilbert transform of w,(t); the Hilbert transform is 
discussed in Appendix 2. To illustrate the nature of this relationship, let 


p(t) = m(t) cos(27f;t) (6.161) 


where m(t) is a band-limited message signal. Typically, the carrier frequency f; is greater 
than the highest frequency component of m(t). Then (in a manner similar to the discussion 
on CAP presented in Section 6.4) the Hilbert transform of y(t) is defined by 


Île) = mlt) sin(2nft) (6.162) 


Since 
T , 
cos( 2af - 2) = sin(27ft) 


we see that ¢;(t) is indeed obtained from y;(t) by shifting the carrier cos(2mft) by —90 
degrees. An important property of Hilbert transformation is that a signal and its Hilbert 
transform are orthogonal to each other. Thus, y;(t) and ĝ;(t) are orthogonal to each other, 
as already stated. 

The average probability of error for the noncoherent receiver of Figure 6.39a is given 
by the simple formula 


1 E 
P, = 7 ep(-357) (6.163) 


where E is the signal energy per symbol, and Nọ/2 is the noise spectral density. 


# DERIVATION OF EQUATION (6.163)* 


To derive the formula of Equation (6.163), we make use of the equivalence depicted in 
Figure 6.39. In particular, we observe that since the carrier phase is unknown, noise at the 
output of each matched filter in Figure 6.394 has two degrees of freedom, namely, in-phase 
and quadrature. Accordingly, the noncoherent receiver of Figure 6.394 has a total of four 
noisy parameters that are conditionally independent given the phase @, and also identically 
distributed. These four noisy parameters have sample values denoted by x14, x91, X12, and 
Xoz; the first two account for degrees of freedom associated with the upper path of Figure 
6.39a, and the latter two account for degrees of freedom associated with the lower path. 

The receiver of Figure 6.39a has a symmetric structure. Hence, the probability of 
choosing s,(¢), given that s,(t) was transmitted, is the same as the probability of choosing 
s,(t), given that s,(t) was transmitted. This means that the average probability of error 
may be obtained by transmitting s(t) and calculating the probability of choosing s2(t), or 
vice versa, assuming that s(t) and s(t) are equiprobable. 

Suppose that signal s(t) is transmitted for the interval 0 = ¢ = T. An error occurs if 
the channel noise w(t) is such that the output /, of the lower path in Figure 6.3 9a is greater 
than the output /, of the upper path. Then the receiver makes a decision in favor of s2(t) 


*Readers who are not interested in the formal derivation of Eq. (6.163) may at this point wish to move on to 
the treatment of noncoherent binary frequency-shift keying (in Section 6.7) and differential phase-shift keying 
(in Section 6.8) as special cases of noncoherent orthogonal modulation, without loss of continuity. 
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FIGURE 6.40 Geometric interpretations of the two path outputs |, and l in the generalized non. 
coherent receiver. 


rather than s,(t). To calculate the probability of error so made, we must have the proba. 
bility density function of the random variable L, (represented by sample value h). Since 
the filter in the lower path is matched to s,(t), and s2(t) is orthogonal to the transmitted 
signal s(t), it follows that the output of this matched filter is due to noise alone. Let xp 
and x» denote the in-phase and quadrature components of the matched filter output in 
the lower path of Figure 6.394. Then, from the equivalent structure depicted in Figure 
6.39b, we see that (for i = 2) 


hL = Vx + xaz (6.164) 


Figure 6.404 shows a geometric interpretation of this relation. The channel noise w(t) is 
both white (with power spectral density No/2) and Gaussian (with zero mean). Corre- 
spondingly, we find that the random variables X;. and Xz (represented by sample values 
Xn and xoz) are both Gaussian-distributed with zero mean and variance No/2, given the 
phase 0. Hence, we may write 


$ 2 
Frxpl%n) = cxo( - z) (6.165) 





and 





1 xo 
Fxoq(*Q2) = Van exo( - in) (6.166) 
Next, we use a well-known result in probability theory, namely, the fact that the envelope 
of a Gaussian process is Rayleigh-distributed and independent of the phase 6 (see Section 
1.12). Specifically, for the situation at hand, we may state that the random variable Lz 
[whose sample value /, is related to xp and xoz by Equation (6.164)] has the following 
probability density function: 


2h (8 
fil) = 4 No P ( N R=? (6.167) 
0, elsewhere 


Figure 6.41 shows a plot of this probability density function. The conditional probability 
that h > h, given the sample value /), is defined by the shaded area in Figure 6.41. Hence; 
we have 


Pp > hll) = f fall) dl (6.168) 
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Ficure 6.41 Calculation of the conditional probability that lL > L, given l. 


Substituting Equation (6.167) into Equation (6.168) and integrating, we get 


i 


Pil, > h|h) = en(-2) (6.169) 


Consider next the output amplitude /,, pertaining to the upper path in Figure 6.394. Since 
the filter in this path is matched to s,(t), and it is assumed that s,(t) is transmitted, it 
follows that |; is due to signal plus noise. Let x, and xp, denote the components at the 
output of the matched filter (in the upper path of Figure 6.39a) that are in phase and in 
quadrature with respect to the received signal, respectively. Then from the equivalent struc- 
ture depicted in Figure 6.39b, we see that (for i = 1) 


= Vai + xb (6.170) 


Figure 6.40b presents a geometric interpretation of this relation. Since a Fourier- 
transformable signal and its Hilbert transform form an orthogonal pair, it follows that xz 
is due to signal plus noise, whereas x is due to noise alone. This means that (1) the 
random variable X;, represented by the sample value xz is Gaussian distributed with mean 
VE and variance No/2, where E is the signal energy per symbol, and (2) the random 
variable Xo, represented by the sample value xo, is Gaussian distributed with zero mean 
and variance Nọ/2. Hence, we may express the probability density functions of these two 
independent random variables as follows: 








_ 1 (xn — VE? 
fx,(%n) VaN, exo( No ) (6.171) 
and 
1 2, 
fxg(Xo1) = VaN, op( 38 ) (6.172) 


Since the two random variables X;, and XQ, are independent, their joint probability den- 
sity function is simply the product of the probability density functions given in Equations 
(6.171) and (6.172). 

To find the average probability of error, we have to average the conditional proba- 
bility of error given in Equation (6.169) over all possible values of /,. Naturally, this 
calculation requires knowledge of the probability density function of random variables L4 
represented by sample value /,. The standard method is now to combine Equations (6.171) 
and (6.172) to find the probability density function of L4 due to signal plus noise. However, 
this leads to rather complicated calculations involving the use of Bessel functions. This 
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analytic difficulty may be circumvented by the following approach. Given xz and x is 
error occurs when, in Figure 6.39, the lower path’s output amplitude /, due to noise rp 
exceeds |, due to signal plus noise; from Equation (6.170) we have 


an 
One 


B = x? + xor (6.173) 


The probability of such an occurrence is obtained by substituting Equation (6.173) into 
Equation (6.169), as shown by 


xh + x2 
E 2) (6.174) 


P(error|xn, x91) = exo( - N 
o 


This is now a conditional probability of error, conditional on the output of the matched 
filter in the upper path taking on values Xn and XQ. This conditional probability mul. 
tiplied by the joint probability density function of Xj and Xo; is then the error-density 
given xn and xo. Since Xy and Xo, are statistically independent, their joint probability 
density function equals the product of their individual probability density functions. The 
resulting error-density is a complicated expression in xy, and xo. However, the average 
probability of error, which is the issue of interest, may be obtained in a relatively simple 
manner. We first use Equations (6.171), (6.172), and (6.174) to evaluate the desired error- 
density as 


Plerror|xn, Xo1)fx,,(*n)f xolo) 

1 1o ened (6.175) 
= Noe "No [xn + xo + (xn — VEY + xoil 
Completing the square in the exponent of Equation (6.175), we may rewrite the exponent 
except for ~1/No as 


VEV E 
x}, + x3, + (xn — VEF + x21 = 2( sn - 5) + 2x + 3 (6.176) 


Next, we substitute Equation (6.176) into Equation (6.175) and integrate the error-density 
over all xy and xg1. We thus evaluate the average probability of error as 


P. = W k P(error|xn, xof xun) f xolo) dxn dxo 


aaa a ae 
=N ap( $) A ex No (>: =) dxn (6.177) 


a 2x2 
: B exo( -22 dxo 


We now use the following two identities: 


i 2 VEV IN, 
T op -4 (xn = 5) | dxy = Sa (6.178) 
> 2 
T exp( —7221) dxo: = “oz (6.179) 


The identity of Equation (6.178) is obtained by considering a Gaussian-distributed variable 
with mean VE/2 and variance N,/4, and recognizing that the total area under the curv 
of a random variable’s probability density function equals unity; the identity of Equatiot 


and 
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(6.179) follows as a special case of Equation (6.178). Thus, in light of these two identities, 
Equation (6.177) simplifies as follows: 


1 E 
P: = 2 eo( 5) 


which is the desired result presented previously as Equation (6.163). 
With this formula at our disposal, we are ready to consider noncoherent binary FSK 
and DPSK as special cases, which we do in the next two sections, respectively.1! 


6.8 Noncoherent Binary 
Frequency-Shift Keying 


In the binary FSK case, the transmitted signal is defined by 


DE, 
= it) 0s:isT, 
sth = dy T, SOOM: sore (6.180) 


0, elsewhere 


where the carrier frequency f; equals one of two possible values, f, and fo; to ensure that 
the signals representing these two frequencies are orthogonal, we choose f. = n;/T,, where 
n; is an integer. The transmission of frequency f represents symbol 1, and the transmission 
of frequency f} represents symbol 0. For the noncoherent detection of this frequency- 
modulated wave, the receiver consists of a pair of matched filters followed by envelope 
detectors, as in Figure 6.42. The filter in the upper path of the receiver is matched to 
cos(27f,t), and the filter in the lower path is matched to cos(2afzt), and in both cases 
0 = t = Te. The resulting envelope detector outputs are sampled at t = T,, and their 
values are compared. The envelope samples of the upper and lower paths in Figure 6.42 
are shown as /, and /,, respectively. Then, if |; > h, the receiver decides in favor of symbol 
1, and if /; < L, it decides in favor of symbols 0. If 1, = h, the receiver simply makes a 
guess in favor of symbol 1 or 0. 

The noncoherent binary FSK described herein is a special case of noncoherent or- 
thogonal modulation with T = T, and E = E,, where T; is the bit duration and E; is the 
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FIGURE 6.42 Noncoherent receiver for the detection of hinary FSK signals. 
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signal energy per bit. Hence, using (Equation (6.163), we find that the bit error rate for 
noncoherent binary FSK is 


pee E ED 
7727P ONG (6.181) 


The formula of Equation (6.181) is derived as a special case of noncoherent orthogonal 
modulation. In Problem 6.31 we address the same issue using a direct approach that 
invokes the application of Rayleigh and Rician distributions; these distributions pertain 
respectively to the random variables Lz and L, whose sample values are defined by Equa. 
tions (6.164) and (6.170), respectively. 


| 6.9 Differential Phase-Shift Keying 


As remarked earlier in Section 6.1, we may view differential phase-shift keying (DPSK) as 
the noncoherent version of PSK. It eliminates the need for a coherent reference signal at 
the receiver by combining two basic operations at the transmitter: (1) differential encoding 
of the input binary wave and (2) phase-shift keying—hence, the name, differential phase- 
shift keying (DPSK). In effect, to send symbol 0, we phase advance the current signal 
waveform by 180 degrees, and to send symbol 1 we leave the phase of the current signal 
waveform unchanged. The receiver is equipped with a storage capability, so that it can 
measure the relative phase difference between the waveforms received during two succes- 
sive bit intervals. Provided that the unknown phase @ contained in the received wave varies 
slowly (that is, slow enough for it to be considered essentially constant over two bit inter- 
vals), the phase difference between waveforms received in two successive bit intervals will 
be independent of 6. 

DPSK is another example of noncoherent orthogonal modulation, when it is 
considered over two bit intervals. Suppose the transmitted DPSK signal equals 
VE,/2T, cos(27f-t) for 0 = t = Te, where T, is the bit duration and E, is the signal 
energy per bit. Let s(t) denote the transmitted DPSK signal for 0 = t = 2T, for the case 
when we have binary symbol 1 at the transmitter input for the second part of this interval, 
namely, T, = t = 2T,. The transmission of symbol 1 leaves the carrier phase unchanged 
over the interval 0 < t = 2T,, and so we define s(t) as 


| Ee zrg 
oT, cos(27f,t), 0<t=T, 


si(t) = (6.182) 


fE 
IT, cos(27ft), Te =t = 2T, 


Let sz(t) denote the transmitted DPSK signal for 0 = t = 2T, for the case when we have 
binary symbol 0 at the transmitter input for T, = t = 2T,. The transmission of 0 advances 
the carrier phase by 180 degrees, and so we define s.(t) as 


a cos(2rf.t), Ost=T, 
s(t) = ” (6.183) 


E 
© cos(2rft + n), Tests 2T, 
IT, 
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We readily see from Equations (6.182) and (6.183) that s4(t) and s(t) are indeed orthog- 
onal over the two-bit interval 0 = £ < 2T,. In other words, DPSK is a special case of 
noncoherent orthogonal modulation with T = 2T, and E = 2E,. Hence, using Equation 
(6.163), we find that the bit error rate for DPSK is given by 


1 


ae cvo( 2] (6.184) 


which provides a gain of 3 dB over noncoherent FSK for the same E;,/No. 


Generation and Detection of DPSK 


The next issue to be considered is the generation of DPSK signals. The differential 
encoding process at the transmitter input starts with an arbitrary first bit, serving as ref- 
erence. Let {d,} denote the differentially encoded sequence with this added reference bit. 
We now introduce the following definitions in the generation of this sequence: 


» If the incoming binary symbol b, is 1, leave the symbol d, unchanged with respect 
to the previous bit. 

> If the incoming binary symbol b, is 0, change the symbol d, with respect to the 
previous bit. 


The differentially encoded sequence {d,} thus generated is used to phase-shift a carrier 
with phase angles 0 and 7 radians representing symbols 1 and 0, respectively. The 
differential-phase encoding process is illustrated in Table 6.7. Note that d, is the comple- 
ment of the modulo-2 sum of b, and d,_,. + 

The block diagram of a DPSK transmitter is shown in Figure 6.43a. It consists, in 
part, of a logic network and a one-bit delay element interconnected so as to convert the 
raw binary sequence {b,} into a differentially encoded sequence [d,}. This sequence is 
amplitude-level encoded and then used to modulate a carrier wave of frequency f., thereby 
producing the desired DPSK signal. 

Suppose next, in differentially coherent detection of binary DPSK, the carrier phase 
is unknown. Then, in light of the receiver being equipped with an in-phase and a quad- 
rature channel, we have a signal space diagram where the received signal points are 
(A cos 8, A sin 0) and (—A cos 9, —A sin 6), with @ denoting the unknown phase and A 
denoting the amplitude. This geometry of possible signals is illustrated in Figure 6.44. The 
receiver measures the coordinates (x;,, xg,) at time t = T, and (x;,, xg,) at time t = 2T,. 
The issue to be resolved is whether these two points map to the same signal point or 
different ones. Recognizing that the two vectors Xp and x,, with end points (x;,, xg,) and 
(x15 Xg,) are pointed roughly in the same direction if their inner product is positive, we 
may formulate the hypothesis test as follows: 


Is the inner product xx, positive or negative? 


i TABLE 6.7 Illustrating the generation of DPSK signal 


{b,} 1 0 0 1 0 0 1 1 

{dy} 1 1 0 1 1 0 1 1 

Differentially encoded 1 1 0 1 1 0 1 1 1 
sequence {d,} 

Transmitted phase 0 0 wr 0 0 m 0 0 0 
(radians) 
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FIGURE 6.43 Block diagrams of (a) DPSK transmitter and (b) DPSK receiver. 


Accordingly, we may write 
say 1 
X11, + Xoo, = 0 (6.185) 
say 0 


We now note the following identity: 


= 
xnžn + Koto, = | [eI + %1,)° — ty ~ xp)? + (og, + xo,  (%Q4 T žo] 


Bie 


Hence substituting this identity into Equation (6.185 ) and multiplying both sides of the 
test by 4, we get the equivalent test: 


say 1 
(xn + Xn + (Xo, + xo — (i) 7 xn) — (Xo — x, 2 0 (6.186) 
say 0 


The decision-making process may therefore be thought of as testing whether the point 
(Xr Xo) 38 closer to (Xr, ¥o,) Or its image (—x1,, —XQ,)- 
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FIGURE 6.44 Signal-space diagram of received DPSK signal. 


Thus, the optimum receiver” for differentially coherent detection of binary DPSK is 
as shown in Figure 6.436, which follows directly from Equation (6.185). This implemen- 
tation merely requires that sample values be stored, thereby avoiding the need for fancy 
delay lines that may be needed otherwise. The equivalent receiver implementation that 
tests squared elements as in Equation (6.186) is more complicated, but its use makes the 
analysis easier to handle in that the two signals to be considered are orthogonal over the 
interval (0, 2T;,); hence, the noncoherent orthogonal demodulation analysis applies. 


6.10 Comparison of Digital Modulation 
Schemes Using a Single Carrier 


@ PROBABILITY OF ERROR 


In Table 6.8 we have summarized the expressions for the bit error rate (BER) for coherent 
binary PSK, conventional coherent binary FSK with one-bit decoding, DPSK, noncoherent 
binary FSK, coherent QPSK, and coherent MSK, when operating over an AWGN channel. 
In Figure 6.45 we have used the expressions summarized in Table 6.8 to plot the BER as 
a function of the signal energy per bit-to-noise spectral density ratio, E,/No. 


è TABLE 6.8 Summary of formulas 
for the bit error rate of different 
_ digital modulation schemes 









Signaling Scheme Bit Error Rate 
(a) Coherent binary PSK 
Coherent QPSK $ erfe(WE,/No) 
Coherent MSK 
(b) Coherent binary FSK $ erfc(WE,/2No) 
(c) DPSK 1 exp(—E,/No) 


(d) Noncoherent binary FSK 4 exp(— E,/2No) 
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FIGURE 6.45 Comparison of the noise performance of different PSK and FSK schemes. 


Based on the performance curves shown in Figure 6.45, the summary of formulas 
given in Table 6.8, and the defining equations for the pertinent modulation formats, we 
can make the following statements: 


1. The bit error rates for all the systems decrease monotonically with increasing values 
of E,/No; the defining curves have a similar shape in the form of a waterfall. 

2. For any value of E;/No, coherent binary PSK, QPSK, and MSK produce a smaller 
bit error rate than any of the other modulation schemes. 

3. Coherent binary PSK and DPSK require an E,/Np that is 3 dB less than the corre- 
sponding values for conventional coherent binary FSK and noncoherent binary FSK, 
respectively, to realize the same bit error rate. 

4. At high values of E,/No,; DPSK and noncoherent binary FSK perform almost as well 
(to within about 1 dB) as coherent binary PSK and conventional coherent binary 
FSK, respectively, for the same bit rate and signal energy per bit. 

5. In coherent QPSK, two orthogonal carriers V2/T cos(27f,t) and V 2/T sin(2aft) 
are used, where the carrier frequency f, is an integer multiple of the symbol rate 
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1/T, with the result that two independent bit streams can be transmitted simulta- 
neously and subsequently detected in the receiver. 

6, In the case of coherent MSK, there are two orthogonal carriers, namely, 
V2/T, cos(2f,t) and V2/T, sin(27f.t), which are modulated by the two antipodal 
symbol shaping pulses cos(t/2T,) and sin(wt/2T,), respectively, over 2T; intervals, 
where T, is the bit duration. Correspondingly, the receiver uses a coherent phase 
decoding process over two successive bit intervals to recover the original bit stream. 

7. The MSK scheme differs from its counterpart, the QPSK, in that its receiver has 
memory. In particular, the MSK receiver makes decisions based on observations over 
two successive bit intervals. Thus, although the transmitted signal has a binary for- 
mat represented by the transmission of two distinct frequencies, the presence of mem- 
ory in the receiver makes it assume a two-dimensional signal space diagram. There 
are four message points, depending on which binary symbol (0 or 1) was sent and 
the past phase history of the FSK signal. 


@ BANDWIDTH EFFICIENCY OF M-ary DIGITAL MODULATION TECHNIQUES 


In Table 6.9, we have summarized typical values of power-bandwidth requirements for 
coherent binary and M-ary PSK schemes, assuming an average probability of symbol error 
equal to 1074 and the systems operating in identical noise environments. This table shows 
that, among the family of M-ary PSK signals, QPSK (corresponding to M = 4) offers the 
best trade-off between power and bandwidth requirements. For this reason, we find that 
QPSK is widely used in practice. For M > 8, power requirements become excessive; ac- 
cordingly, M-ary PSK schemes with M > 8 are not as widely used in practice. Also, co- 
herent M-ary PSK schemes require considerably more complex equipment than coherent 
binary PSK schemes for signal generation or detection, especially when M > 8. (Coherent 
8-PSK is used in digital satellite communications.) 

Basically, M-ary PSK and M-ary QAM have similar spectral and bandwidth char- 
acteristics. For M > 4, however, the two schemes have different signal constellations, For 
M-ary PSK the signal constellation is circular, whereas for M-ary QAM it is rectangular. 
Moreover, a comparison of these two constellations reveals that the distance between the 
message points of M-ary PSK is smaller than the distance between the message points of 
M-ary QAM, for a fixed peak transmitted power. This basic difference between the two 
schemes is illustrated in Figure 6.46 for M = 16. Accordingly, in an AWGN channel, 
M-ary QAM outperforms the corresponding M-ary PSK in error performance for M > 4. 


TABLE 6.9 Comparison of power-bandwidth 
requirements for M-ary PSK with binary 
PSK. Probability of symbol error = 10~* 






(Bandwidth) mary (Average power) ary 
Value of M (Bandwidth) ginary (Average power) binary 





4 0.5 0.34 dB 
8 0.333 3.91 dB 
16 0.25 8.52 dB 
32 0.2 13.52 dB 








From Shanmugan (1979, p. 424). 
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FIGURE 6.46 Signal constellations for (a) M-ary PSK and (b) corresponding M-ary QAM, for 
M= 16. 


However, the superior performance of M-ary QAM can be realized only if the channel js 
free of nonlinearities. 

As for M-ary FSK, we find that for a fixed probability of error, increasing M results 
in a reduced power requirement. However, this reduction in transmitted power is achieved 
at the cost of increased channel bandwidth. In other words, M-ary FSK behaves in an 
opposite manner to that of M-ary PSK. We will revisit this issue in an information- 
theoretical context in Chapter 9, and thereby develop further insight into the contrasting 
behaviors of M-ary PSK and M-ary FSK. 


i 6.11 Voiceband Modems 


The “modem,” a contraction of the term modulator-demodulator, is a conversion device 
that facilitates the transmission and reception of data over the public switched telephone 
network (PSTN). The data of interest may be digital signals generated by computers or 
service providers. In such an application, the modulator portion of the modem converts 
the incoming digital signal into a standard form suitable for transmission over a telephone 
channel in the PSTN. The demodulator portion of the modem receives the channel output 
and reconverts it into the original digital signal format. In yet another application, namely, 
fax modems, or more precisely modems with facsimile capability, the data may represent 
text, graphics, pictures, or combinations thereof, In this latter application, the document 
of interest is coded into a series of compressed picture elements (pixels), which are then 
transmitted over the telephone channel by modulating their values according to a prede- 
fined modulation standard. When the fax modem is in a receiving mode of operation, the 
demodulator portion of the modem. operates on the received analog signal and decom- 
presses the corresponding binary data representation of the demodulated signal into a neat 
or actual duplicate of the original transmitted image. In what follows, we focus our atten- 
tion on modems that provide communication between a user and an Internet Service Pro- 
vider (ISP) over the PSTN. 

Traditionally, the PSTN has been viewed as an analog network. In reality, howevets 
the PSTN as we presently know it has become an almost entirely digital network. In most 
cases, the only part of the PSTN that has remained analog (and will likely remain so for 
many years to come) is the local loop, which represents the relatively short connection 
from a home to the central office. Thus, depending on how the PSTN is used, we may 
identify two distinct classes of modem configurations, symmetric and asymmetric, a$ ws 
scribed next. 
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SYMMETRIC MODEM CONFIGURATIONS 


The simplest approach to the design of modems is to treat the entire PSTN as a linear 
analog network, as indicated in Figure 6.47. (Recall from Chapter 3 that the PSTN is 
almost entirely digital due to the use of pulse-code modulation (PCM) for the transmission 
of voice signals.) In such a setting, analog-to-digital and digital-to-analog conversions are 
needed whenever the modems send signals to and receive signals from the PSTN. The 
modem configuration depicted in Figure 6.474 exhibits “symmetry” in that both modems 
are identical and the data rate downstream (from the ISP to the user) is exactly the same 
as the data rate upstream (from the user to the ISP). 

The symmetric modem configuration of Figure 6.47a embodies a large number of 
modem types, ranging in data rate from 300 b/s to 36,600 b/s, as summarized in Table 
A6.7 on a selection of standard modems. The design of modems began with frequency- 
shift keying, which catered to relatively low data rates. As the demand for data transmis- 
sion over telephone channels increased, increasingly more sophisticated modulation tech- 
niques were employed to better use the information capacity of the telephone channel. 

Consider, for example, the popular V.32 modem standard that.has the following 
characteristics: 


Carrier frequency = 1,800 Hz 
Modulation rate = 2,400 bauds 
Data rate = 9,600 b/s 


The signaling data rate of 9,600 b/s assumes a high signal-to-noise ratio. The V.32 standard 
specifies two alternative modulation schemes: 


Nonredundant coding. Under this scheme, the incoming data stream is divided into 
quadbits (i.e., groups of four successive bits) and then transmitted over the telephone 
channel as 16-QAM. In each quadbit, the most significant input dibit undergoes 
phase modulation, whereas the least significant input dibit undergoes amplitude 
modulation. Discussing the phase modulation first, practical considerations favor 
the use of differential phase modulation for the receiver need only be concerned with 
the detection of phase charges. This matter is taken care of by using a differential 
encoder, which consists of a read-only memory and a couple of delay units, as 
shown in Figure 6.48a. Let Q,,,,Q2,,, denote the current value of the most significant 
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Ficure 6.47 (a) Environmental overview of symmetric modem configuration: the upstream and 
downstream data rates are equal. (b) Environmental overview of “asymmetric” modem configura- 
tion: data rate downstream is higher than upstream. 
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FIGURE 6.48 Block diagrams of V.32 modem. (a) Nonredundant coding. (b) Trellis coding. 


input dibit, and let J,,,-1J2,.-1 denote the previous value of the corresponding dibit 
output by the encoder. Then, in response to the dibits Q1,,Q2,, and Ty .-an-1 the 
differential encoder produces the dibit J,,,,J2,,, which, in turn, induces a phase change 
in the modulated signal. This phase change, measured in the counterclockwise di- 
rection, is governed by the Gray coding scheme of Table 6.10. Note that the phase 
change is determined entirely by the input dibit Q1,,Q2,»- Insofar as the differential 
phase modulation is concerned, there is one other matter that needs to be addressed: 
a code for identifying the four quadrants of the two-dimensional signal space. This 
second matter is resolved by adopting the Gray coding scheme included in Figure 
6.49. 

Turning next to the amplitude modulation, a code has to. be specified for the 
four possible values which the least significant input dibit, denoted by Q3,„Qa,m 64" 
assume in, say, the first quadrant. This matter is taken care of by adopting the Gray 
code for the four signal points in the first quadrant shown lightly shaded in Figure 
6.49. 

The final issue that needs to be resolved is the 90° rotational invariance, which 
is mandated by the use of differential encoding. This form of invariance means that 
the overall M-ary QAM constellation looks exactly the same when it is rotate 


TABLE 6.10 Phase changes 
induced by differential encoding 
in the V.32 modem due to varying 
input dibits 






Current input dibit 


6.11 


Voiceband Modems 


423 





Phase change 
Qan (degrees) 
0 0 90 
0 1 0 
1 0 180 
1 1 270 





through an integer multiple of 90 degrees, regardless of whether it is coded or un- 
coded; then the receiver can correctly decode the transmitted message sequence when 
the local oscillator phase differs from the carrier phase by an integer multiple of 90 
degrees. This final requirement is satisfied by filling in the Gray codes for the signal 
points in the remaining three quadrants in the manner shown in Figure 6.49. Dashed 
arrows are included in Figure 6.49 to illustrate the 90° rotational invariance. 

Putting all of these matters together for the combined amplitude and phase 
modulation, we get the 16-QAM constellation shown previously in Figure 6.174, 
which is reproduced here as Figure 6.50a. Correspondingly, the encoding system 
consists of a differential encoder followed by a 16-point signal-space mapper, as 
shown in Figure 6.484. The V.32 modem so configured is said to be nonredundant 
because, with 16 constellation points, the transmitted 4-bit code word has no redun- 
dant bits. 


$i 





FIGURE 6.49 Illustrating the Gray encoding of the four quadrants and dibits in each quadrant 
for the V.32 modem. The dashed arrows illustrate the 90° rotational invariance. 
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FIGURE 6.50 (a) Signal constellation of V.32 modem using nonredundant coding. (b) Signal 
constellation of V.32 modem using trellis coding. 


As an illustrative example of how this particular V.32 modem operates, let the cur- 
rent group of four input bits be 1001 and the dibit previously output by the modem be 
11. For this example, we thus have 


Q;,,Q2,, = 10 
Q3,nQa,n = 01 
L,n-1l2,7-1 =11 


Then in light of the coding scheme for the four quadrants specified in Figure 6.49, the 
previous output dibit 11 means that the modulator was previously residing in the first 
quadrant. Because the corresponding input dibit is 10, it follows from Table 6.10 that the 
modulator experiences a phase change of 180° in the counterclockwise direction, thereby 
switching its operation into the third quadrant identified by the dibit 00. Finally, with the 
current value of the least significant dibit Q;,,Qu4,, being 01, the modulator outputs a 
QAM signal whose coordinates are a, = —3 (along the ġ;-axis) and 6, = —1 (along the 
ġ2-axis). This output corresponds to the code word 0001. 

When the signal-to-noise ratio is not high enough, the V.32 modem switches to its 
QPSK mode, operating at the reduced rate of 4,800 b/s. In this latter mode of operation, 
the four states of the modem are signified by the points labeled A, B, C, and D in Figure 
6.504. 


Trellis Coding 


Trellis coding is a forward-error correction scheme where coding and modulation 
are treated as a combined entity rather than as two separate operations. Figure 6.48 
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shows the encoding system of the V.32 modem with trellis coding. The incoming data 
stream is divided into quadbits, but unlike the case of nonredundant coding, they are 
transmitted over the channel as a 32-QAM signal. 

As indicated in Figure 6.486, the trellis encoder involves the use of a convolu- 
tional encoder, which operates on the output of the differential encoder. (Convolutional 
encoders are discussed in Chapter 10.) However, the choice of convolutional encoding is 
restricted in the V.32 modem to accommodate the use of differential encoding (i.e., 90 
degrees rotational invariance). Indeed, this requirement cannot be satisfied by a linear 
convolutional encoder. Rather, the convolutional encoder must be #onlinear;"* see Prob- 
lem 10.30. 

The data-encoding process in the V.32 modem with trellis coding proceeds in three 
stages: 


1. The differential encoder in Figure 6.48b, in response to the current input dibit 
Q1,nQ2,, and the previous differentially encoded dibit I „- vo, 1-1, produces the dibit 
Th plone 

2. The differentially encoded current dibit J,,,J2,,, is input to the convolutional encoder 
in Figure 6.48b, which produces a three-bit output. One of these bits is a parity- 
check bit, denoted by Yo,„. The value of Yo,,, depends on the other two bits, Yı „and 
Y2,», produced by the convolutional encoder. 

3. The bits Yon, Yı,„ and Y2,,, produced by the convolutional encoder, together with 
the least significant input dibit Q3,„Q4,„ are applied to the signal-space mapper in 
Figure 6.48b, which selects one of the states in the 32-point constellation shown in 
Figure 6.50b as the modem output. 


The parity-check bit Yo,,, provides a modem with trellis coding better immunity to 
channel impairments than a V.32 modem with nonredundant coding, an advantage that 
is gained without an increase in bandwidth requirements. In quantitative terms, trellis 
coding provides an effective coding gain of 4 dB compared to 16-QAM. Coding gain 
expresses how much more signal energy per data bit is needed by the uncoded modem for 
the same level of noise performance. 

However, for this advantage of trellis coding to be realized in practice, the signal- 
to-noise ratio must be high enough. Otherwise, the V.32 modem is switched to its 
QPSK mode of operation, which is signified by the four states labeled A, B, C, and D 
in Figure 6.50b. In this latter mode of operation, the data rate of the modem is reduced 
to 4,800 b/s. 


= ASYMMETRIC MODEM CONFIGURATIONS 


For a more efficient use of the PSTN, we should treat it as what it really is: an almost 
entirely digital network that is nonlinear. In particular, since the ISP is digitally imple- 
mented, the need for analog-to-digital conversion at the ISP modem is eliminated. This 
means that the communication between the ISP and the PSTN can be entirely digital, as 
portrayed in Figure 6.47b. However, the user’s modem has to remain analog because the 
local loop is analog. This, in turn, requires the use of analog-to-digital and digital-to-analog 
conversions each time the user’s modem sends signals to and receives signals from the 
PSTN. The modem configuration depicted in Figure 6.476 is “asymmetric” in that it is 
possible for the downstream signaling data rate to be much higher than the upstream 
signaling data rate, as explained next. 
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As mentioned earlier, a digital PSTN is based on the use of PCM for the transmission 
of voice signals. Features of the system relevant to the present discussion are as follows 
(see Chapter 3): 


» Data signaling rate of 64 kb/s, which is made up of a sampling rate of 8 kHz ang 
the representation of each voice sample by an 8-bit code word, 

> Fifteen-segment companding law (e.g., a logarithmic u-law with u = 255) for com. 
pressing the voice signal at the transmitter and expanding it at the receiver, 


From the discussion on PCM presented in Chapter 3 we also recall that quantization only 
affects analog-to-digital conversion but not digital-to-analog conversion. These obserya. 
tions have a profound impact on the optimum strategy for the design of asymmetric 
modems. 

Suppose there is no analog-to-digital conversion between a digital modem at the Isp 
and the digital portion of the PSTN, and the digitally connected transmitter of the modem 
is designed to properly use the nonuniformly spaced 256 (discrete) threshold levels of the 
digital PSTN. Then, since digital-to-analog conversion is completely unaffected by quan- 
tization noise, it follows that the information transmitted by the ISP’s digital modem 
reaches the user’s analog modem with no loss whatsoever. On the basis of these arguments, 
in theory, it should be possible to transmit data from the ISP to the user at a rate equal to 
the 64 kb/s data rate of the digital PSTN. But system limitations inherent to the PSTN 
reduce the attainable data-rate down to 56 kb/s, as explained in the sequel. 


Digital Modem 
From the description of a PCM voiceband channel presented in Chapter 3, we find 


that the design of the digital modem is constrained by three factors not under our control. 
The design constraints are: 


1. A sampling rate f, = 8 kHz. 

2. A set of M = 256 allowable threshold levels built into the construction of the com- 
pressor (i.e., transmitter portion of the compander). 

3. A baseband (antialiasing) filter of about 3.5 kHz bandwidth, built into the front end 
of the PCM transmitter. 


In light of these constraints, we may now state the fundamental philosophy underlying the 
design of the digital modem as follows: 


Design a signal s(t) at the digital modem’s input such that each of its samples taken 
at the rate f = 8 kHz matches one of the M = 256 threshold levels of the com- 
pressor, and the transmitted signal satisfies Nyquist’s criterion for zero intersymbol 
interference. 


(Nyquist’s criterion for zero intersymbol interference was discussed in Chapter 4.) 


One Realization of the Digital Modem 


A solution to this signal design problem is made particularly difficult by the fact that 
the PCM transmit filter has a bandwidth of about 3.5 kHz and not 4 kHz (half the sampling 
rate f). The immediate implication of this constraint is that instead of the desired set © 
8,000 samples, we can only generate 2 X 3,500 = 7,000 independent samples every seco" 
in accordance with Nyquist’s criterion for zero intersymbol interference. How then do we 
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Ficure 6.51 Group of N uniformly spaced samples, repeating every (N + 1)T, seconds. 


fit 7,000 independent samples per second within the prescribed framework of 8,000 sam- 
ples per second? 

To answer this fundamental question, we make use of the recurrent nonuniform 
equivalent form of the sampling theorem. To be more specific, consider the situation de- 
picted in Figure 6.51, where the samples are divided into groups, with each group con- 
taining N uniformly spaced samples, and the groups having a recurrent period of (N + 1)T, 
seconds, where T; = 1/f. The illustration presented in Figure 6.51 is for the problem at 
hand: T; = 125 us and N = 7. The sampling instants in the nonuniform distribution of 


Figure 6.51 are written as 

thi = te + (N + IDIT, (6.187) 
k=1,2,...,N 
l= 0, +1, 42,... 


= (k — 1)T; + (N + 1\IT, 


The stage is now set for us to define the band-limited signal s(t) as follows: 


o0 N 
s(t) = = 2, s(te ailt — (N + 1)IT,) (6.188) 


where the interpolation function yy (t) is itself defined by 


g : sol e=) 
hto) = sine —" ) i - 


(N + 1)T, q=l , T 
ark Sin NFD (th — t3) 





(6.189) 


Computing Equation (6.189) for N = 7, we obtain the seven standard pulses plotted in 
Figure 6.52, where time is normalized with respect to the sampling period T;. These pulses 
exhibit the following properties: l 


» Each standard pulse is normalized so that we have 
u(S) =wik-1)=1 fork =1,2,...,7 


Note, however, that the peak of the kth pulse does not occur at time th = (k — 1)T,. 
> Fork =1,2,...,7 the pulse yy (¢/T,) goes through zero at times t + (k — 1)T; modulo 
(N + 1), except at those times that are congruent to t = (—1) modulo (N + 1). 


Accordingly, the signaling scheme for the digital modem consists of a recurrent non- 
uniform pulse amplitude modulation scheme. The amplitudes of seven uniformly spaced 
samples in each group of eight samples are determined by the incoming data stream and 
in conformity to the threshold levels of the compressor in the PCM transmitter. In effect, 
these seven samples are the independent samples that are responsible for carrying the 
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FIGURE 6.52 A digital modem’s waveforms of the standard pulses y(t), k = 1, 2,..., 7. 


incoming data stream across the PSTN every 1,000 us. Moreover, they deliver the data to 
the receiver with zero intersymbol interference. The remaining “eighth” samples are com- 
pletely determined by the independent samples and known beforehand to the system; they 
do not carry information and are therefore discarded at the receiver. Thus the digital 
modem is capable of transmitting digital data across the PSTN almost errorless at a rate 
equal to 56 kb/s, which is calculated as follows: 


7 X 1,000 x log, 256 = 56,000 b/s 


One last comment is in order. The standard pulses Y(t) can be constructed so as to 
decay at a rate faster than 1/t. To do so, we simply replace the sinc function in Equation 
(6.189) by a Nyquist pulse with a rolloff in a manner similar to that described in Chap- 
ter 4. 


Another Realization of the Digital Modem 


The kind of digital modem just described is bidirectional, assuming that both ends 
of the data link are analog. However, a simpler solution to the digital modem design 
problem ensues when one end of the link is digital and asymmetric data rates are possible. 

Consider what happens when a data sequence consisting of octets (i.e., 8-bit code 
words) arrives at the PSTN. There they will be treated as octets representing speech er 
coded in accordance with the u-law or A-law, depending on the part of the world where 
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the PSTN is located. Consequently, the D/A converter, which drives the analog modem, 
produces a continuous-time signal defined by 

s(t) = 3 alag — kT.) (6.190) 

k 

where c, is the kth octet in the data sequence, a(c,) is the representation level specified by 
the pertinent companding law, T; is the sampling interval (equal to 125 us), and g(t) is an 
interpolation function bandlimited to a frequency below 1/2T,, or about 4 kHz, to satisfy 
the reconstruction part of the sampling theorem; see Section 3.2. 

In the normal operation of the PSTN, the signal s(t) represents a reconstructed speech 
signal. However, in the case of input data, s(t) appears like noise. In any event, from a 
communication theoretical perspective, the signal s(t) in Equation (6.190) may be viewed 
as a pulse-amplitude modulated signal. Herein lies the theoretical basis for the design of 
the digital modem. Specifically, the design is based on a signal constellation as in an analog 
modem, except that the constellation is constructed from one-dimensional PCM symbols 
rather than two-dimensional QAM symbols. 

Ordinarily, the data rate achievable by a digital modem is limited to about 56 kb/s 
because of the following factors: 


1, The inner levels of the compander in the PSTN are very closely spaced, as shown in 
Table 3.4; hence they are susceptible to residual intersymbol interference and noise 
following the modem’s equalizer. 

2. Least significant bits (LSBs) are robbed from the data stream for various purposes 
internal to the PSTN; this “bit-robbing” can be as much as (but usually less than) 
8 kb/s and always in a periodic pattern. 


Analog Modem 


Unlike the digital modem, the noise performance of the analog modem is limited 
essentially by quantization noise in the u-law or A-law governing the operation of the 
PCM compander. Typically, the signal-to-noise ratio on a good PCM voiceband channel 
is on the order of 34 to 38 dB. The other channel impairment that limits the operation of 
the analog modem is the effect of bandlimiting imposed by the antialiasing and interpo- 
lation filters, which, as already mentioned, is typically about 3.5 kHz. 

A sophisticated choice for the analog modem is the standard V.34 modem, which 
operates at rates extending up to 33.6 kb/s. The fundamental design philosophy of this 
modem embodies five distinctive features.” 


1. 960-QAM super-constellation. 

The signal constellation is said to be a super- or nested-constellation in that it consists of 
four constellations: the QAM constellation shown in Figure 6.53 with 240 message points, 
and its rotated versions through 90, 180, and 270 degrees. 


2. Adaptive bandwidth. 

The transmitter probes the channel by sending a set of tones, which permits measurement 
of the signal-to-noise ratio at the channel output as a function of frequency. The modem 
is thereby enabled to select the appropriate carrier frequency and bandwidth according to 
the probing results and available symbol rates. 


3, Adaptive bit rates. 

During the training of the receiver, the bit rate is selected according to the receiver’s esti- 
mate of the maximum bit rate, which the modem can support at bit error rates as low as 
10~¢ to 107%. 
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FIGURE 6.53 Quarter-superconstellation of V.34 modem with 240 signal points. The full super- 


constellation is obtained by combining the rotated versions of these points by 0, 90, 180, and 270 
degrees. (Taken from Forney et al., 1996, with permission of the JEEE.) 


4. Trellis coding. 

This error-control coding technique is used to provide an effective coding gain of about 
3.6 dB; there is an optional more powerful trellis code with an effective coding gain of 
about 4.7 dB. 


5. Decision feedback equalization. 

To make full use of the available telephone channel bandwidth, including frequencies near 
the band edges where there can be attenuation as much as 10 to 20 dB, a decision feedback 
equalizer (DFE) is used. (The DFE is discussed in Chapter 4.) However, it is nota straight- 
forward matter to combine coding with DFE because decision feedback requires immediate 
decisions, whereas coding inherently involves decoding delay. The overcome this problem, 
the feedback section of the DFE is moved to the transmitter, which is made possible 
through the use of the Tomlinson-Harashima precoding. (This form of equalization 14 
precoding is discussed briefly in Note 12 of Chapter 4.) 
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V.90 Modem 


The V.90 modem standard embodies digital and analog modems. The digital modem 
at the ISP end is based on the second realization described earlier; it sends data downstream 
at the rate of 56 kb/s. The analog modem at the user’s end is a V.34 modem standard, 
transmitting data upstream at the rate of 33.6 kb/s. These two highly different rates confirm 
the asymmetric nature of the V.90 modem. j 

The outstanding feature of the V.90 modem, namely, the downstream data rate of 
56 kb/s, makes it suitable for use on the Internet for downloading graphics in intensive 
Web pages, audio, and video at near-ISDN speeds. 


l 6.12 Multichannel Modulation 


The asymmetric digital subscriber line (ADSL), described in Section 4.8, is a data trans- 
mission system capable of realizing megabit rates over existing twisted-pair telephone lines. 
Specifically, ADSL runs at a downstream data rate up to 9 Mb/s and an upstream data 
rate up to 1 Mb/s. These data signaling rates fit the access requirements of the Internet 
perfectly. (As mentioned in Section 4.8, the upstream bit rate should be about 10 percent 
of the downstream bit rate for efficient operation of the Internet protocol.) The challenge 
in designing ADSL is to develop a line code that exploits the information capacity of the 
channel as fully as possible. The carrierless amplitude phase modulation (CAP), discussed 
in Section 6.4, provides one approach for solving this difficult passband data transmission 
problem. Another approach is to use an equally elegant modulation technique called dis- 
crete multitone. This latter approach is a form of multichannel modulation'® that allows 
the modulator characteristics to be a function of measured channel characteristics. It is 
fitting that we begin the discussion by describing multichannel modulation, which we do 
in this section, followed by discrete multitone in the next section, 

The basic idea of multichannel modulation is rooted in a commonly used engineering 
principle: divide and conquer. According to this principle, a difficult problem is solved by 
dividing it into a number of simpler problems, and then com bining the solutions to those 
simple problems. In the context of our present discussion, the difficult problem is that of 
data transmission over a wideband channel with severe intersymbol interference, and the 
simpler problems are exemplified by data transmission over AWGN channels. We may 
thus summarize the essence of multichannel modulation as follows: 


Data transmission over a difficult channel is transformed through the use of ad- 
vanced signal processing techniques into the parallel transmission of the given data 
stream over a large number of subchannels, such that each subchannel may be 
viewed effectively as an AWGN channel. 


Naturally, the overall data rate is the sum of the individual data rates over the subchannels 
operating in parallel. 


u CAPACITY OF AWGN CHANNEL 


From the Background and Preview material presented in the opening chapter, we recall 
that, according to Shannon’s information capacity theorem, the capacity of an AWGN 
channel (that is free from intersymbol interference) is defined by 


C = B log,(1 + SNR) b/s (6.191) 
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where B is the channel bandwidth, and SNR denotes the signal-to-noise ratio measure d 
the channel output. A proof of this important theorem is formally presented in Chate 
9, For now it suffices to say that for a given SNR, we can transmit data over an AWGN 
channel of bandwidth B at the maximum rate of C bits per second with arbitrarily small 
probability of error, provided that we employ an encoding system of sufficiently high 
complexity. Equivalently, we may express the capacity C in bits per transmission or chan- 
nel use as 


1 ; ai 
C= 3 log.(1 + SNR) bits/transmission (6.192) 


In practice, we usually find that a physically realizable encoding system must transmit 
data at a rate R less than the maximum possible rate C for it to be reliable. For an imple. 
mentable system operating at low enough probability of symbol error, we thus need to 
introduce a signal-to-noise ratio gap or just gap, denoted by T. The gap is a function of 
the permissible probability of symbol error P, and the encoding system of interest. It pro. 
vides a measure of the “efficiency” of an encoding system with respect to the ideal trans. 
mission system of Equation (6.192). With C denoting the capacity of the ideal encoding 
system and R denoting the capacity of the corresponding implementable encoding system, 
the gap is defined by 


Peal 
T= 
SNR (6.193) 
T21 
Equivalently, we may write 
R= Hiog(1 + SR) bits/transmission (6.194) 


For encoded PAM or QAM operating at P, = 1076, for example, the gap T is constant at 
8.8 dB. Through the use of codes (e.g., trellis codes discussed in Chapter 10), the gap F 
may be reduced to as low as 1 dB. 

Let P denote the transmitted signal power, and o” denote the channel noise variance 
measured over the bandwidth B. The signal-to-noise ratio is therefore 


P 
NR = > 
S z 
where 
o = NoB 


We may thus finally define the attainable data rate as 
1 P 
Ze eile i issi 6.195) 
R > og: 1 + T :) bits/transmission ( 


With this formula at hand, we are ready to describe multichannel modulation in quant 
tative terms. 


a CONTINUOUS-TIME CHANNEL PARTITIONING 


Consider a linear wideband channel (e.g., twisted pair) with an arbitrary frequency Ag 
sponse H(f). Let the squared magnitude response | H(f)| be approximated by a staircas? 
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FIGURE 6.54 Staircase approximation of an arbitrary magnitude response | H(f) |; only positive- 
frequency portion of the response is shown. 


function as illustrated in Figure 6.54, with Af denoting the width of each step. In the limit, 
as the frequency increment Af approaches zero, the staircase approximation of the channel 
approaches the actual H(f). Along each step of the approximation, the channel may be 
assumed to operate as an AWGN channel free from intersymbol interference. The problem 
of transmitting a single wideband signal is thereby transformed into the transmission of a 
set of narrowband orthogonal signals. Each narrowband orthogonal signal, with its own 
carrier, is generated using a spectrally efficient modulation technique such as M-ary QAM, 
with additive white Gaussian noise being essentially the only primary source of transmis- 
sion impairment. This, in turn, means that data transmission over each subchannel of 
bandwidth Af can be optimized by invoking Shannon’s information capacity theorem, 
with the optimization of each subchannel being performed independently of all the others. 
Thus, in practical signal-processing terms, the need for complicated equalization of a wide- 
band channel is replaced by the need for multiplexing and demultiplexing the transmission 
of the incoming data stream over a large number of narrowband subchannels that are 
contiguous and disjoint. Although the resulting complexity of a multicarrier system is 
indeed high for a large number of subchannels, implementation of the entire system can 
be accomplished in a cost-effective manner through the use of VLSI technology. 

Figure 6.55 shows a block diagram of the multichannel data transmission system in 
its most basic form. The system is configured here using quadrature-amplitude modulation 
whose choice is justified by virtue of its spectral efficiency. The incoming binary data stream 
is first applied to a demultiplexer (not shown in the figure), thereby producing a set of N 
substreams. Each substream represents a sequence of two-element subsymbols, which, for 
the symbol interval 0 = t = T, is denoted by 


(an, Dy), n=1,2,...,N 


where a, and b,, are element values along the two coordinates of subchannel n. 
Correspondingly, the passband basis functions of the quadrature-amplitude modu- 
lators are defined by the function pairs 


($ (t) cos(2af,t), P(t) sin(27f,t)], n= 1,2,...,N (6.196) 


where the carrier frequency f,, of the nth modulator is an integer multiple of the symbol 
rate 1/T, as shown by 


fn Fo n=1, 2... N 
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Figure 6.55 Block diagram of multichannel data transmission system. 


and the low-pass function ¢ (t) is the sinc function: 


(t) = faé) , ES (6.197) 


The passband basis functions defined here have the following desirable properties (see 
Problem 6.41 for their proofs): 


Property 1 


For each n, the two quadrature-modulated sinc functions form an orthogonal pair as 
shown by 


ie (b(t) cos(2f,,t))(H(t) sin(2af,t)) dt = 0 for all n (6.198) 


oo 


This orthogonal relationship provides the basis for formulating the signal constellation for 
each of the N modulators in the form of a squared lattice. 
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Property 2 
Recognizing that 
exp(j27f,t) = cos(2af,,t) + j sin(2zf,t) 


we may completely redefine the passband basis functions in the complex form 
l e(t) explants), n=1,2,...,N (6.199) 


where the factor 1/\/2 has been introduced to ensure that the scaled function ẹ(t)/ V2 
has unit energy. Hence, these passband basis functions form an orthonormal set, as shown 


by 


is (5 g(t) ex wmte\(4, a(t) e (anh) dt = f ane (6.200) 
ae V2 P n V2 XP{ JLT ]e 0, kén 3 


where the asterisk denotes complex conjugation. 


Equation (6.200) provides the mathematical basis for ensuring that the N modulator- 
demodulator pairs operate independently of each other. 


Property 3 


The set of channel-output functions (h(t) X  (t)| remains orthogonal for a linear channel 
with arbitrary impulse response h(t), where % denotes convolution. 


The channel is thus partitioned into a set of independent subchannels operating in contin- 
uous time. 

Figure 6.55 also includes the structure of the receiver. It consists of a bank of N 
coherent detectors, with the channel output being simultaneously applied to the detector 
inputs. Each detector is supplied with a locally generated pair of quadrature modulated 
sinc functions operating in synchrony with the pair of passband basis function applied to 
the corresponding modulator in the transmitter. 

Each subchannel may have some residual intersymbol interference (ISI). However, 
as the number of subchannels N approaches infinity, the ISI disappears. Thus, for a 
sufficiently large N, the bank of coherent detectors in Figure 6.55 operates as maximum 
likelihood detectors, independently of each other and ona subsymbol-by-subsymbol basis. 

To define the detector outputs in response to the input subsymbols, we find it con- 
venient to use complex notation. Let A, denote the subsymbol applied to the mth modulator 
during the symbol interval 0 = ż < T: 


An = dy + jbm on =1,2,...,N (6.201) 
The corresponding detector output is 
Y,=H,A,+W, 2=1,2,...,N (6.202) 
where H, is the complex-valued frequency response of the channel evaluated at the sub- 
channel carrier frequency f = f,: 
H, = Hf) 2=1,2,...,N (6.203) 


The W, is a complex-valued random variable due to the channel noise w(t); the real and 
imaginary parts of W,, have zero mean and variance N/2. With knowledge of the mea- 
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sured frequency response H(f) available, we may therefore use Equation (6.202) to com. 
pute a maximum likelihood estimate of the transmitted subsymbol A,,. The estimates A 
A,,..., Ân so obtained are finally multiplexed to produce the corresponding estimate of 
the original binary data transmitted during the interval 0 =¢ = T. 

To summarize, for a sufficiently large N, we may implement the receiver as an op- 
timum maximum likelihood detector, operating as N subsymbol-by-subsymbol detectors 
The reason why it is possible to build a maximum likelihood receiver in such a simple way 
is the fact that the passband basis functions constitute an orthonormal set, and their op- 
thogonality is maintained for any channel impulse response b(t). 


gs GEOMETRIC SIGNAL-TO-NOISE RATIO 


In the multichannel transmission system of Figure 6.55, each subchannel is characterized 
by a SNR of its own. It would be highly desirable to derive a single measure for the 
performance of the entire system of Figure 6.55. 

To simplify the derivation of such a measure, we assume that all of the subchannels 
in Figure 6.55 are represented by one-dimensional constellations. Then the channel ca. 
pacity of the entire system in bits per transmission is given by 





1 
12 P, 
= ON > lors(1 + za) 
i (6.204) 


il P, 
= oN logs i (1 + es] 


1 N P, 1N 
=5 toes TI (: + fs) 


Let (SNR)overat denote the overall signal-to-noise ratio of the entire system. We may then 
express R in bits per transmission as 


R= ; log. (1 + Nese) (6.205) 








T 
Comparing Equations (6.205) with (6.204), we may thus write 


N UN 
(SNR)ovecat = (i (: + Pe) > 1) (6.206) 


n=1 To; 





Assuming that P,,/T'a@? is high enough to ignore the two unity terms in Equation (6.206), 
we may approximate the overall SNR as 


N /p 1N 
(SNR) = I] (z) (6.207) 


n=1 7. 


We may thus characterize the overall system by a SNR that is the geometric mean of the 
SNRs of the individual subchannels. F 

The geometric SNR of Equation (6.207) can be improved considerably by distrib- 
uting the available transmit power among the N subchannels on a nonuniform basis. This 
objective is attained through the use of loading as discussed next. 
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LOADING OF THE MULTICHANNEL TRANSMISSION SYSTEM 


Equation (6.204) for the bit rate of the entire multichannel system ignores the effect of the 
channel on system performance. To account for this effect, define 


=|H(f,)|, 2=1,2,...,N (6.208) 


Then assuming that the number of subchannels N is large enough, we may assume that 
&, is constant over the entire bandwidth Af assigned to subchannel » for all #. In such a 
case, we may modify the second line of Equation (6.204) for the overall SNR of the system 
as g 


= EPn 
R= D low + e2) (6.209) 
The g3 and T are usually fixed. The noise variance 72 is AfNo for all n, where Af is the 
bandwidth of each subchannel and N)/2 is the noise power spectral density. We may 
therefore optimize the overall bit rate R through a proper allocation of the total transmit 
power among the various channels. However, for this optimization to be of practical value, 
we must maintain the total transmit power at some constant value P, say, as shown by 


N 
> P, = P = constant (6.210) 
n=1 
The optimization we therefore have to deal with is a constrained optimization problem, 
which may be stated as follows: 


Maximize the bit rate R for the entire multichannel transmission system through 
an optimal sharing of the total transmit power P between the N subchannels, subject 
to the constraint that P is maintained constant. 


To solve this optimization problem, we first use the method of Lagrange multipliers’? to 
set up an objective function that incorporates the constraint of Equation (6.210), as shown 
by 





rete S 
-AÈ oss(1 + si) + a(p -> P,) 


n=1 


am! S EAW S 
= JN lowe 2 log (1 + gis) + a(P > Pr 


n m=1 


(6.211) 





where A is the Lagrange multiplier. Hence, differentiating J with respect to P„, then setting 
the result equal to zero and finally rearranging terms, we get 


1 
2N 


p, + I2 


loge 
=A (6.212) 





7 
This result indicates that the solution to our constrained optimization problem is to have 


2 
Pat =K forn = 1,2,...,N (6.213) 


n 
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where K is a prescribed constant under the designer’s control. That is, the sum of th 
transmit power and the noise variance (power) scaled by the ratio T/gŻ must be maintained 
constant for each subchannel. The process of allocating the transmit power P to the in. 
dividual subchannels so as to maximize the bit rate of the entire multichannel transmission 
system is called loading. 


WATER-FILLING INTERPRETATION OF THE OPTIMIZATION PROBLEM 


In solving the constrained optimization problem just described, two conditions must be 
satisfied, namely, Equations (6.210) and (6.213). The optimum solution so defined has an 
interesting interpretation as illustrated in Figure 6.56 for N = 6, assuming that the gap T 
is constant over all the subchannels. To simplify the illustration in Figure 6.56 we have set 
a2 = No Af = 1, that is, the average noise power is unity for all N subchannels. Referring 
to this figure, we may now make the following observations: 


» The sum of power P,, allocated to channel and the scaled noise power '/g? satisfies 
the constraint of Equation (6.213) for four of the subchannels for a prescribed value 
of the constant K. 

> The sum of power allocations to these four subchannels consumes all the available 
transmit power, maintained at the constant value P. 

> The remaining two subchannels have been eliminated from consideration because 
they would each require negative power to satisfy Equation (6.213) for the prescribed 
value of the constant K; this condition is clearly unacceptable. : 


The interpretation illustrated in Figure 6.56 prompts us to refer to the optimum solution 
of Equation (6.213), subject to the constraint of Equation (6.210), as the water-filling 
solution. This terminology follows from analogy of our optimization problem with a fixed 
amount of water (standing for transmit power) being poured into a container with a 
number of connected regions, each having a different depth (standing for noise power), 
The water distributes itself in such a way that a constant water level is attained across the 
whole container. We have more to say on the water-filling interpretation of information 
capacity in Chapter 9. 

Returning to the task of how to allocate the fixed transmit power P among the 
various subchannels of a multichannel transmission system so as to optimize the bit rate 


Energy 











index of subchannel, n 


FIGURE 6.56 Water-filling interpretation of the loading problem. 
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of the entire system, we may proceed as follows. Let the total transmit power be fixed at 
the constant value P as in Equation (6.210). Let K denote the constant value prescribed 
for the sum P, + 'a2/g2 for all n as in Equation (6.213). We may then use this pair of 
equations to set up the following system of simultaneous equations: 


P, +P, + Py =P 


P, -K = —To*/g? 
P -K = -Tolg? (6.214) 
Py — K = -Tolg 


where we have a total of (N + 1) unknowns and (N + 1) equations to solve for them. We 
may rewrite this set of simultaneous equations in matrix form as 


11-1 0 Py P 

10 0 -1] | P —To*/gt 

01 0 -1 =| —Tolg3 (6.215) 
pot poi Py : 

00 = 1 -1]| K] | regi 


Premultiplying both sides of Equation (6.215) by the inverse of the (N + 1)-by-(N + 1) 
matrix on the left-hand side of the equation, we obtain solutions for the unknowns P,, 
P,...5 Px, and K. We should always find that K is positive, but it is possible for some 
of the Ps to be negative. The negative Ps are discarded as power cannot be negative. 


i» EXAMPLE 6.7 


Consider a linear channel whose squared magnitude response | H(f) |? has the piecewise-linear 
form shown in Figure 6.57. To simplify the example, we set the gap T = 1 and the noise 
variance g? = 1, In the situation so described, the application of Equation (6.214) yields 


P,+P,=P 
P, -K=-1 
P,~K=-1/ 
IHI? 

1.0 





f 
f -fı 0 h f 


FIGURE 6.57 Squared magnitude response for Example 6.7. 
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Figure 6.58 Water-filling profile for Example 6.7. 


where the total transmit power P is normalized with respect to the noise variance. Solving 
_ these three simultaneous equations for P4, P}, and K, we get 


Since 0 < I < 1, it follows that P, > 0, but it is possible for P, to be negative. This latter 
condition can arise if 


1 


< 
i PAT 





But then P, exceeds the prescribed value of transmit power P. It follows therefore that in this 
example the only acceptable solution is to have 1/(P + 1) < } < 1. Suppose then we have 
P = 10 and / = 0.1, for which the solution is 


K= 10.5 
Py = 9.5 
P, = 0.5 


The corresponding water-filling picture is portrayed in Figure 6.58. 


i 6.13 Discrete Multitone 








The material presented in Section 6.12 provides an insightful introduction to the notion 
of multichannel modulation. In particular, the continuous-time channel partitioning 1- 
duced by the passband basis functions of Equation (6.196) or equivalently (6.199) exhibits 
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a highly desirable property: Orthogonality of the basis functions (and therefore the channel 
partitioning) is preserved despite their convolution with the impulse response of the chan- 
nel. However, the system has two shortcomings: 


1. The passband basis functions use a sinc function that is nonzero for an infinite time 
interval, whereas practical considerations favor a finite observation interval. 

2. For a finite number of subchannels, N, the system is suboptimal; optimality of the 
system is assured only when N approaches infinity. 


We may overcome these shortcomings by using discrete multitone (DMT), the basic 
idea of which is to transform a wideband channel into a set of N subchannels operating 
in parallel. What makes DMT distinctive is the fact that the transformation is performed 
in discrete time as well as discrete frequency. Consequently, the transmitter input-output 
behavior of the entire communication system admits a linear matrix representation, which 
lends itself to implementation using the discrete Fourier transform. 

To explore this new approach, we first recognize that in a realistic situation the 
channel has its nonzero impulse response, /(t), essentially confined to a finite interval 
[0, T,]. So, let the sequence 4g, hy, . . . , h, denote the baseband equivalent impulse response 
of thechannel sampled at the rate 1/T,, with 


T, =(1+ 07, (6.216) 


The sampling rate 1/T, is chosen to be greater than twice the higher frequency component 
of interest in accordance with the sampling theorem. To continue with the discrete-time 
description of the system, let s[”] = s(#T,) denote a sample of the transmitted symbol s(ż), 
win] = w(nT,) denote a sample of the channel noise w(t), and x[n] = x(nT,) denote the 
corresponding sample of the channel output (received signal). The channel performs linear 
convolution on the incoming symbol sequence {s[7]} of length N, producing a channel 
output sequence {x[7]} of length N + v. Extension of the channel output sequence by v 
samples compared to the channel input sequence is due to the intersymbol interference 
produced by the channel. 

To overcome the effect of intersymbol interference, we create a cyclically extended 
guard interval whereby each symbol sequence is preceded by a periodic extension of the 
sequence itself. Specifically, the last v samples of the symbol sequence are repeated at the 
beginning of the sequence being transmitted, as shown by 


s[—k] = s[N — k] fork =1,2,...,» (6.217) 
This condition is called a cyclic prefix. The excess bandwidth factor due to the inclusion 
of the cyclic prefix is therefore v/N, where N is the number of transmitted samples after 


the guard interval. 
With the cyclic prefix in place, the matrix description of the channel takes the form 


x[N-1] bo bh, ba = bai Bee 0 = 0 JF s(N-1) wIN-1] 
x(N — 2] 0 ho hy boa bya by 0 s(N—-2) w[N-2] 
x[N-v-1] =-|0 0 0 0 bo h = b, s[N-v-1] + Bet (6.218) 


0 Ore 
x[N~v~2] h, 0- 0 0 ba bma || sIN-v-2] w{N—-v—-2] 


D 


x[0] ie oe ae ho <{0] w/0] 


Equivalently, we may describe the discrete-time representation of the channel in the com- 
pact matrix form 


x=Hs+w (6.219) 
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wW 


FIGURE 6.59 Discrete-time representation of multichannel data transmission system, 


where the transmitted symbol vector s, the channel noise vector w, and the received signal 
vector x are all N-by-1 vectors which are respectively defined by 


s = [s[N — 1], s[N — 2],..., s[0]]” (6.220) 

w = [w[N ~ 1], w[N - 2],..-, [Ol (6.221) 
and 

x = [x[N — 1], x[N — 2],..., x[0]]” (6.222) 


We may thus depict the discrete-time representation of the channel as in Figure 6.59. The 
N-by-N channel matrix H is defined by 


b hi ha = ha h OD 
0 bo bh = bnz bea by = 0 

H=|0 0 0 = 0 b h= h (6.223) 
bh 0 0 = 0 0 h = bya 
hy bp h = bh, 0 0 = b 


From this definition, we readily see that the matrix H has the following structural com- 
position: Every row of the matrix is obtained by applying a right-shift to the previous row 
by one position, with the added proviso that the rightmost element of the previous row 
spills over in the shifting process to be “circulated” back to the leftmost element of the 
new row. Accordingly, the matrix H is referred to as a circulant matrix. 

Before proceeding further, it is befitting that we briefly review the discrete Fourier 
transform and its role in the spectral decomposition of the circulant matrix H. 


# DISCRETE FOURIER TRANSFORM 


Consider the N-by-1 vector x of Equation (6.222). The discrete Fourier transform (DFT) 
of the vector x is defined by the N-by-1 vector 


X = [XIN - 1], XIN — 2],..., XI0]]" (6.224) 


where 


N-1 
X[k] = x 2 x[n] cxp( =i 27 en), k=0,1,...,N—1 (6225) 


The exponential term exp(—j2kn/N) is referred to as the kernel of the DFT. Correspond- 
ingly, the inverse discrete Fourier transform (IDFT) of the N-by-1 vector X is defined by 


ie 
x[n] = © X[k] exp jZ kn), n=0,1,...,N-1 (628 
VN &o N 
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Although Equations (6.225) and (6.226) appear to be similar, they have different inter- 
pretations. Given the signal vector x, Equation (6.225) provides a spectral representation 
of the signal computed at a set of discrete frequencies: f, = k/N, which are normalized 
with respect to the sampling rate. Given the transformed vector X, Equation (6.226) re- 
covers the original signal vector x. We may therefore view Equation (6.225) as the analysis 
equation and Equation (6.226) as the synthesis equation. 

An important property of a circulant matrix, exemplified by the channel matrix H 
of Equation (6.223), is that it permits spectral decomposition as shown by 


H = QAQ (6.227) 


where the superscript * denotes Hermitian transposition (i.e., the combination of complex 
conjugation and ordinary matrix transposition). Descriptions of the matrices Q and A are 
presented in the sequel in that order. 

The matrix Q is a square matrix defined in terms of the kernel of the N-point DFT 
as follows: 


oxn( -j 27 a — ON = n) oo eop- aN - ») esp( -1 22 0N - n) 1 


exo(- = (N -1N — 2) a exo( -1 27 20N a 2) exo(-i = (N - 2) 1 
=A : : ; 
1 m 1 1 1 


(6.228) 


From this definition, we readily see that the k/th element of the N-by-N matrix, Q, starting 
from the bottom right at k = 0 and / = 0 and counting up step-by-step, is 


1 2 
wu = Gye EH), ik, D) =0,1,...,N—-1 (6.229) 
The matrix Q is an orthonormal matrix or unitary matrix in that it satisfies the condition 


QQ =I (6.230) 


where I is the identity matrix. That is, the inverse matrix of Q is equal to the Hermitian 
transpose of Q. 

The matrix A is a diagonal matrix that contains the N discrete Fourier transform 
values of the sequence ho, /1,..., 4, characterizing the channel. Denoting these transform 
values by An-1,---,A1, Ào, We May express A as 


dept 0 is 0 
Oo ieee: Sec D 

A=| , eo (6.231) 
0 0 wad No. 


(The As here are not to be confused with the Lagrange multipliers in Section 6.12.) 
The DFT has established itself as one of the principal tools of digital signal processing 
by virtue of its efficient computation using the fast Fourier transform (FFT) algorithm”? 
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Specifically, the FFT algorithm requires on the order of N log, N operations rather tha 

the N? operations for direct computation of the DFT. For efficient implementation of i 
FFT algorithm, we should choose the block length N an integer power of two. The ia 
putational savings obtained by using the FFT algorithm are made possible by exploitin, 

the special structure of the DFT defined in Equation (6.225). Moreover, these savings 
become more substantial as we increase the data length N. 


FREQUENCY-DOMAIN DESCRIPTION OF THE CHANNEL 
With this brief description of the DFT on hand, we are ready to resume our discussion of 
discrete multitone. First, we define 

s = Q'S (6.232) 


where S is the frequency-domain vector representation of the transmitter input. Each ele- 
ment of the N-by-1 vector S$ may be viewed as a complex-valued point in a two-dimen- 
sional QAM signal constellation. Given the channel output vector x, we define its corre- 
sponding frequency-domain representation as 


X = Qx (6.233) 


Using Equations (6.227), (6.232) and (6.233), we may rewrite Equation (6.219) in the 
equivalent form 


X = Q(QIAQQ'S + W) (6.234) 
Hence, using the relation of Equation (6.230), we simply get 
X=AS+W (6.235) 
where 
W = Qw (6.236) 


In expanded form, Equation (6.235) reads as 
Xp = AS, + Wis k=0,1,...,N-1 (6.237) 


where the set of frequency-domain values (AJE is known for a prescribed channel. 

For a channel with additive white noise, Equation (6.237) implies that the receiver 
is composed of a set of independent processors operating in parallel. With the A, all known, 
we may thus use the block of frequency-domain values {X,}fco' to compute estimates of 
the corresponding transmitted block of frequency domain-values Shio. 


e DFT-BAasED DMT SYSTEM 


Equations (6.235), (6.225), (6.226), and (6.237) provide the mathematical basis for the 
implementation of DMT using the DFT. Figure 6.60 illustrates the block diagram of the 
system derived from these equations and their practical implications. 

The transmitter consists of the following functional blocks: 


> Demultiplexer, which converts the incoming serial data stream into parallel form. 

> Constellation encoder, which maps the parallel data into N/2 multibit subchannels 
with each subchannel being represented by a QAM signal constellation. Bit allocation 
among the subchannels is also performed here in accordance with a loading 
algorithm. 
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FIGURE 6.60 Block diagram of the discrete-multitone (DMT) data-transmission system. 


» Inverse discrete Fourier transformer (IDFT), which transforms the frequency-domain 
parallel data at the constellation encoder output into parallel time-domain data. For 
efficient implementation of the IDFT using the fast Fourier transform (FFT) algo- 
rithm, we need to choose N = 2* where & is a positive integer. 

> Parallel-to-serial converter, which converts the parallel time-domain data into serial 
form. Guard intervals stuffed with cyclic prefixes are inserted into the serial data on 
a periodic basis before conversion into analog form. 

» Digital-to-analog converter (DAC), which converts the digital data into analog form 
ready for transmission over the channel. 


Typically, the DAC includes a transmit filter. Accordingly, the time function h(t) should 
be redefined as the combined impulse response of the cascade connection of the transmit 
filter and the channel. 

The receiver performs the inverse operations of the transmitter, as described here: 


> Analog-to-digital converter (ADC), which converts the analog channel output into 
digital form. 

> Serial-to-parallel converter, which converts the resulting bit stream into parallel form. 
Before this conversion takes place, the guard intervals (cyclic prefixes) are removed. 

® Discrete Fourier transformer (DFT), which transforms the time-domain parallel data 
into frequency-domain parallel data; as with the IDFT, the FFT algorithm is used to 
implement the DFT. 
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> Decoder, which uses the DFT output to compute estimates of the original multi-bit 
subchannel data supplied to the transmitter. 

> Multiplexer, which combines the estimates so computed to produce a reconstruction 
of the transmitted serial data stream. 


APPLICATIONS OF DMT 


An important application of DMT is in the transmission of data over two-way channels 
Indeed, DMT has been standardized for use on asymmetric digital subscriber lines (ADSL3) 
using twisted pairs. The ADSL was described in Chapter 4. For example, DMT provides 
for the transmission of data downstream (i.e., from an Internet service provider to a sub. 
scriber) at the DS1 rate of 1.544 Mb/s and the simultaneous transmission of data upstream 
(i.e., from the subscriber to the Internet service provider) at 160 kb/s. This kind of data 
transmission capability is well suited for handling data-intensive applications such aş 
video-on-demand. 

DMT is also a core technology in implementing the asymmetric very-high-rate digital 
subscriber lines?’ (VDSL), which differs from all other DSL transmission techniques be. 
cause of its ability to deliver extremely high data rates. For example, VDSL can provide 
data rates of 13 to 26 Mb/s downstream and 2 to 3 MB/s upstream over twisted pairs that 
emanate from an optical network unit and connect to the subscriber over distances less 
than about 1 km. These high data rates allow the delivery of digital TV, super-fast Web 
surfing and file transfer, and virtual offices at home. 

The use of DMT for ADSL and VDSL provides a number of advantages: 


> The ability to maximize the transmitted bit rate, which is provided by tailoring the 
distribution of information-bearing signals across the channel according to channel 
attenuation and noise conditions. 

» Adaptivity to changing line conditions, which is realized by virtue of the fact that 
the channel is partitioned into a number of subchannels. 

& Reduced sensitivity to impulse noise, which is achieved by spreading its energy over 
the many subchannels of the receiver. As the name implies, impulse noise is chat- 
acterized by long, quiet intervals followed by narrow pulses of randomly varying 
amplitude. In an ADSL or VDSL environment, impulse noise arises due to switching 
transients coupled to wire pairs in the central office and to various electrical devices 
on the user’s premises. 


COMPARISON OF DIGITAL SUBSCRIBER LINES AND VOICEBAND MODEMS 


In Section 6.11 we discussed voiceband modems that are already close to operating at their 
theoretical limits of 33.6 kb/s upstream and 56 kb/s downstream. In this section we have 
discussed the application of DMT to VDSLs that can operate at data rates of about 2 to 
3 Mb/s upstream and 13 to 26 Mb/s downstream. These two vastly different sets of up- 
stream/downstream data rates prompt the following question: How is it possible for VDSL 
to operate at rates about three orders of magnitude faster than voiceband modems ovet 
the same twisted pairs (i.e., phone lines)? The reason for this vast difference in operating 
data rates between voiceband modems and VDSLs is not the twisted pairs; rather, it is the 
digital switches built into a public switched telephone network that prevent the transport 
of broadband data to subscribers (users) via voiceband modems. Simply put, the digita 
switches treat digital data in the same way as voice signals for which they are primar! 7 
designed. 
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FIGURE 6.61 (a) Voiceband modem environment. (b) xDSL (digital subscriber line) environ- 
ment, where x stands for “asymmetric” or “very high-rate.” 


Figure 6.61 highlights the operational environments of voiceband modems and 
xDSLs, where x stands for A in ADSL and V in VDSL. In the model of Figure 6.614 
pertaining to a voiceband modem, we have a relatively long transmission path between an 
Internet service provider (ISP) and a subscriber. Most importantly, the transmission path 
traverses through a narrowband public switched telephone network (PSTN), which limits 
the available channel bandwidth to about 3.5 kHz. In contrast, in the model of Figure 
6.615 pertaining to xDSL, the transmission path accommodates the transport of broad- 
band data between the ISP and subscriber via a broadband integrated services digital 
network and a relatively short local loop consisting of a twisted pair. The system permits 
the coexistence of POTS and xDSL signals on the same local loop, which is made possible 
through the use of a pair of splitters, as indicated in Figure 6.610; splitters, consisting of 
bidirectional low-pass and high-pass filters, are discussed in Section 4.8. 


ORTHOGONAL FREQUENCY DIVISION MULTIPLEXING” 


Discrete multitone is one particular discrete form of multichannel modulation. Another 
closely related form of this method of modulation is orthogonal frequency-division mul- 
tiplexing (OFDM) that differs from DMT in areas of application and some aspects of its 
design. 

OFDM is used for data transmission over radio broadcast channels and wireless 
communication channels. This domain of application requires some changes to the design 
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of the OFDM system. Unlike DMT that uses loading for bit allocation, OFDM use, 
fixed number of bits per subchannel. This restriction is made necessary by the fact that. 
broadcast channel involves one-way transmission, and in a wireless communications en 
vironment the channel is varying too rapidly. Accordingly, in both cases it is not feasible 
for the transmitter to know the channel and how to “load” it. 

Thus, the block diagram of Figure 6.60 applies equally to OFDM except for the fact 
that the signal constellation encoder does not include a loading algorithm for bit allocation 
In addition, two other changes have to be made to the design of the system: i 


> In the transmitter, an upconverter is included after the digital-to-analog converter tg 
translate the transmitted frequency, thereby facilitating the propagation of the trans. 
mitted signal over a radio channel. 

> In the receiver, a downconverter is included before the analog-to-digital converter to 
undo the frequency translation that was performed by the upconverter in the 
transmitter. 


Applications of OFDM include the following: 


1. Wireless communications. 
OFDM, combined with coding and interleaving, provides an effective technique to combat 
multipath fading that is a characteristic feature of wireless communication channels, 


2. Digital audio broadcasting. 
OFDM has been adopted as the standard for digital audio broadcasting in Europe. Here 
again the system involves the combined use of coding and interleaving. 


(Error-control coding and related issues are discussed in Chapter 10.) 


§ 6.14 Synchronization 


The coherent reception of a digitally modulated signal, irrespective of its form, requites 
that the receiver be synchronous to the transmitter. We say that two sequences of events 
(representing a transmitter and a receiver) are synchronous relative to each other when the 
events in one sequence and the corresponding events in the other occur simultaneously. 
The process of making a situation synchronous, and maintaining it in this condition, is 
called synchronization.” 

From the discussion presented on the operation of digital modulation techniques, we 
recognize the need for two basic modes of synchronization: 


1. When coherent detection is used, knowledge of both the frequency and phase of the 
carrier is necessary. The estimation of carrier phase and frequency is called carrier 
recovery or carrier synchronization. 

2. To perform demodulation, the receiver has to know the instants of time at which 
the modulation can change its state. That is, it has to know the starting and finishing 
times of the individual symbols, so that it may determine when to sample and whea 
to quench the product-integrators. The estimation of these times is called clock re- 
covery or symbol synchronization. 


These two modes of synchronization can be coincident with each other, or they can occut 
sequentially one after the other. Naturally, in a noncoherent system, carrier synchroniza- 
tion is of no concern. 
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Synchronization can be implemented in one of two fundamentally different ways: 


1. Data-aided synchronization. 

In data-aided synchronization systems, a preamble is transmitted along with the data- 
bearing signal in a time-multiplexed manner on a periodic basis. The preamble contains 
information about the carrier and symbol timing, which is extracted by appropriate pro- 
cessing of the channel output at the receiver. Such an approach is commonly used in digital 
satellite and wireless communications, where the motivation is to minimize the time re- 
quired to synchronize the receiver to the transmitter. Its limitations are two-fold: (1) re- 
duced data-throughput efficiency that is incurred by assigning a certain portion of each 
transmitted frame to the preamble, and (2) reduced power efficiency by allocating a certain 
fraction of the transmitted power to the transmission of the preamble. 


2. Nondata-aided synchronization. 

In this second approach, the use of a preamble is avoided, and the receiver has the task of 
establishing synchronization by extracting the necessary information from the modulated 
signal. Both throughput and power efficiency are thereby improved but at the expense of 
an increase in the time taken to establish synchronization, 


In any event, synchronization is basically a statistical parameter estimation problem. 
A principled approach for solving such a problem is maximum likelihood estimation (see 
Section 5.5), which proceeds by first formulating a log-likelihood function of the parameter 
of interest given the received signal. This formulation is relatively straightforward by treat- 
ing the channel noise as a Gaussian process. Most important, it requires no prior infor- 
mation about the modulated signal. 

In this section we confine our attention to nondata-aided forms of carrier and timing 
synchronization systems. In this context, we may identify two approaches for solving the 
synchronization problem, given a modulated signal with suppressed carrier to conserve 
power: 


1. Classical approach. 

An essential building block in the classical approach to synchronization is the phase-locked 
loop. (The phase-locked loop was discussed in Chapter 2.) Specifically, for carrier recovery 
the receiver requires the use of a suppressed-carrier tracking loop for providing a coherent 
secondary carrier (subcarrier) reference. For example, we may use a variant of the Costas 
loop or the Mth power loop for M-ary PSK. The standard Costas loop for double sideband- 
suppressed carrier (DSB-SC) modulation was discussed in Chapter 2. As for the Mth power 
loop, it consists of the cascade connection of an Mth power-law device, band-pass filter, 
phase-locked loop, and frequency divider by M. The objective here is to exploit the ac- 
quisition and tracking properties of the phase-locked loop. For further discussion of the 
Mth power loop, the reader is referred to Problem 6.47. 


2. Algorithmic (modern) approach. 

In the modern approach, the solution to maximum likelihood estimation is formulated in 
algorithmic form using discrete-time signal processing. Specifically, implementation of the 
synchronizer is built on an algorithm that provides an estimate of carrier phase or symbol 
timing on an iteration-by-iteration basis. The processing is performed in the baseband 
domain to pave the way for the use of discrete-time (digital) signal processing. 


In this section we describe the algorithmic approach to synchronization for M-ary PSK 
systems for both carrier recovery and symbol-timing recovery. 
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The approach taken in the exposition is sequential in that timing recovery is per 
formed before phase recovery. The reason for so doing is that if we know the group fa 
incurred by transmission through the channel, then one sample per symbol at the matched 
filter output in the receiver is sufficient for estimating the unknown carrier phase. More. 
over, the computational complexity of the receiver is minimized by using synchronization 
algorithms that operate at the symbol rate 1/T. 


a DECISION-DIRECTED RECURSIVE ALGORITHM FOR PHASE RECOVERY 


As remarked earlier, the first important step in solving the synchronization problem is to 
formulate the log-likelihood function for the carrier phase 9, given the Gaussian noise- 
contaminated received signal. Let /(@) denote this log-likelihood function, which serves as 
the objective function for estimating 9. The next step is to determine the derivative of H4) 
with respect to 6. The final step is to formulate a recursive (iterative) algorithm for com. 
puting a maximum likelihood estimate of the unknown @ in a step-by-step manner. 


Evaluation of 91(0)/0 0 
Let s,(t) denote the transmitted signal for symbol k = 0, 1,...,M— 1: 
sult) = | costaft +a), 0547 (6.238 
where E is the symbol energy, T is the symbol period, and 
2a 27 
a = 0, 570019 MDPM (6.239) 
Equivalently, we may write 
s(t) = E cos(2rf.t + a,)g(t) (6.240) 


where g(t) is the shaping pulse, namely, a rectangular pulse of unit amplitude and duration 
T. Let 7, denote the carrier (phase) delay, and 7, denote the envelope (group) delay, both 
of which are introduced by the channel. By definition, 7, affects the carrier and 7, affects 
the envelope. Then the received signal is 


x(t) = = cos(2 rft — Te) + ærg(t — Tg) + w(t) 
(6.241) 


= E cos(2af.t + 0 + a,)g(t — Te) + w(t) 


where w(t) is the channel noise and 8 is defined as —27f.7, to be consistent with the 
notation in Section 6.6. Both the carrier phase @ and group delay 7, are unknown. How- 
ever, it is assumed that they remain constant over the observation interval 0 = £ $ To or 
through the transmission of Lo = To/T symbols. Equivalently, we may write (using 710 
place of 7, to simplify matters) 


2E 
x(t) = = cos(2af-t + 0 + ær) + w(t), rztasT+r (6.242) 


* A reader who is not interested in the formal derivation of ol(#\/a@ may omit this subsection a nd move onto the 
next subsection without loss of continuity. 
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At the receiver the basis functions are defined by 


lt) = fe cos(2 rf), TstsTt?t (6.243) 
dlt) = E sin(2rft) retsTt+r (6.244) 


Here it is assumed that the receiver has perfect knowledge of the carrier frequency fa; 
otherwise, a carrier frequency offset has to be included, which complicates the analysis. 
Accordingly, we may represent the received signal x(t) by the vector 


x(7) = bee (6.245) 
x2(7) 
where 
T+r , 
x(t) = Í x(t)ġ;(t) dt, 7=1,2 (6.246) 


In a corresponding fashion, we may express the signal component of x(£) by the vector 


54g 0, sf 


S2(dp > ð, 7) 


s(x, 0, T) = | (6.247) 
where a, is the transmitted symbol and 


T+r 
Silar 6, T) = Í E cos(2mf.t + 0 + a,)¢,(t) dt fori =1,2 (6.248) 


Assuming that f, is an integer multiple of the symbol rate 1/T, we have 


silap, 0, T) = VE cos(@ + œ) (6.249) 
S2(az, 0, 7) = —VE sin(0 + az) (6.250) 
We may thus write 
X;.(7) = S(az; 4, 7) +w (6.251) 
where w is the noise vector 
w= A l (6.252) 
w2 
with 
TEF 
w= Í wlte) dt, i= 1,2 (6.253) 


The w; is the sample value of a Gaussian random variable W of zero mean and variance 
N,/2, where No/2 is the (two-sided) power spectral density of the channel noise w(t). 

The conditional probability density function of the random vector X, given the trans- 
mission of symbol a, and the occurrence of carrier phase @ and group delay 7, is 





= 1 1 2 
fx(x| ae, 8, 7) 7N, exo N, || xe(z) — s(az, 8, 7) | ) (6.254) 
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For a, = 0 the received signal x(t) equals the channel noise w(t), so 


1 1 
fx(x|a, = 0) = 7N, exp( N, I| xx(7) | :) (6.255) 


Hence we may define the likelihood function for M-ary PSK at the receiver as 





=e Frxl&| des 0, 7) 


L{a;, 0, T) iag frxlx| ay _ 0) 
a. Zoe 1 s (6.256) 
= op xglT)Slar, 8, 7) — No || slar, 0, 7) || ) 
In M-ary PSK, 
|| s(2z, 8, 7) | = constant 


as the message points lie on a circle of radius VE. Hence, ignoring the second term in the 
exponent, we may simplify the likelihood function as 


2 
L(a,, 8, 7) = eo = xz(7)s(ay, 0, a) (6.257) 
0 
Assuming that we transmit a sequence of Lo statistically independent symbols, namely, 
a = [do lis. o +5 Argal” (6.258) 
the resulting likelihood function is 
Lo—1 A 
La, 6,7) = [I opf x} (7)s(ax, 8, ») (6.259) 
k=0 No 


The log-likelihood function is therefore 
Ka, 0, 7) = log L(a, 0, 7) 
2 471 (6.260) 
=N Š, XE (T)S(4x, O, 7) 
From Equations (6.249) and (6.250) we deduce 


8, (8) = s(ap, 4, 7) 


~ vE| Sate] k=0,1,..., Lo- 1 


—sin(â, + 6) 


(6.261) 


where â, is an estimate of the actual a, produced at the detector output for the symbol 
a,. Correspondingly, we may express the matched filter output as 


Xi, 
xX, = 
TX2k 


Hence, using this definition and Equation (6.261) in Equation (6.260), we get 
2VE 45} 


N; p [x1 cos(& + 0) + x2, sin(âr + 0)] 
_2VE W (6.262) 


> [(x1, cos & + x2, sin&,) cos 8 
No ko 


1(8) 





— (xi, sin & — x2, cos &) sin 6] 


6.14 Synchronization 453 


Differentiating /(6) with respect to 8, we obtain 


ala) — 2VE 


—— = > [(x1, cos & + X2 sin âp) sin 0 
k=0 


oe No (6.263) 
+ (x1 sind — x2, cos &) cos 8] 
We may simplify Equation (6.263) by introducing the following notations: 
Xp = Xip + jxrp (6.264) 
and 
ae (6.265) 


cos a, + j sina, 


where <, is the complex envelope (i.e., baseband value) of the matched filter output due 
to the kth transmitted symbol, and a, is a symbol indicator in the message constellation 
of the M-ary PSK. We may thus write 


Re[4zXe] = Re[(cos dy — j sin &)(x14 + j*2,)] 








A a (6.266) 
= X14 COS A, + X24 sin dy 
Im[âk č] = Iml(gosak= j sin da). + jX2,)] (6.267) 
= —X1 k SIN A, + X2, COSA, 
We may also note from Euler’s formula: 
e`} = cos6 — j sin 0 (6.268) 
Accordingly, we may rewrite Equation (6.263) in the compact form: 
alo) 2 VE 43' Sass dg = 
Fe = ND, (Refaim) + (maze) (Refe-*)) 
2VE Lo-1 (6.269) 
= 5 Im[4zz,e| 
No kæ 


where â, is an estimate of a,, and the asterisk denotes complex conjugation. 


m RECURSIVE ALGORITHM FOR MAXIMUM LIKELIHOOD ESTIMATION 
OF THE CARRIER PHASE 


With the formula of Equation (6.269) for the derivative of the log-likelihood function (6) 
with respect to the carrier phase 6 at hand, we are now ready to formulate an algorithm 
that seeks to maximize /(@). We would like to perform the maximization in an iterative 
fashion so that the receiver is enabled to respond to the received signal on a symbol-by- 
symbol basis. To that end, we may build on the following algorithmic idea borrowed from 
adaptive filtering (see the discussions on the LMS algorithm presented in Chapters 3 


and 4): 
(aa _ ( oid ) 2 baer eee (6.270) 
estimate estimate parameter, \ signal 


where the error signal, or the adjustment signal to be more precise, is defined as the 
instantaneous value of the gradient of the log-likelihood function /(#) with respect to 8. 
Note that the parameter adjustment applied to the old estimate in Equation (6.270) is 
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FIGURE 6.62 Recursive Costas loop. 


positive as the objective here is to perform gradient ascent. From Equation (6.269) we 
readily see that the error signal (i.e., the instantaneous value of 01(6)/36 due to the trans. 
mission of a single symbol) is given by 


eln] = Im[472,e7"] (6.271) 
where the scaling factor 2\/E/Np is accounted for in what follows. Also, we have used n 


in place of k to denote a time step or iteration of the algorithm. Accordingly, we use 
Equation (6.270) to write 


6[n + 1] = O[n] + yeln] (6.272) 


where 6[7] is the old estimate of the carrier phase 6, 6[n + 1] is the updated estimate of 
6, and yis the step-size parameter; the scaling factor 2VE/Np is absorbed in y. 
Equations (6.271) and (6.272) define the recursive algorithm for phase recovery. This 
algorithm is implemented using the system shown in Figure 6.62, which may be viewed 
as a recursive generalization of the Costas loop. We may therefore refer to it as the recursive 
Costas loop for phase synchronization. 
The following points should be noted in Figure 6.62: 


> The detector supplies an estimate of the transmitted symbol a,, given the matched 
filter output. 


> The look-up table supplies the value of exp(— j6[n]) = cos ĝia] — sin ôf] for an 
input 6[x]. 
> The output of the error generator is the error signal e[7]. 


œ The block labeled z~? is a unit-delay element with the delay equal to the symbol 
period T. 


The recursive Costas loop of Figure 6.62 uses a first-order digital filter. To improve 
the tracking performance of this synchronization system we may use a second-order digita 
filter. Figure 6.63 shows an example of a second-order digital filter made up of a cascade 
of two first-order sections, with p as an adjustable loop parameter. An important property 
of a second-order filter used in the Costas loop for phase recovery is that it will eventually 
lock onto the incoming carrier with no static error, provided that the frequency em or 
between the receiver and transmitter is initially small. 
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FIGURE 6.63 Second-order digital filter. 


NONDATA-AIDED RECURSIVE ALGORITHM FOR SYMBOL TIMING 


For timing synchronization the only assumption made is that the receiver has knowledge 
of the carrier frequency f.. The requirement is to develop an algorithm for recursive esti- 
mation of the group delay 7 incurred in the course of transmitting the modulated signal 
through the channel. 

Let L(a,, 6, 7) denote the likelihood function of 7, which is also a function of trans- 
mitted symbol a, and carrier phase 6. The likelihood function is defined by Equation 
(6.257). To proceed further we must-remove the dependencies of L(a,, 0, 7) on the trans- 
mitted data sequence {a,} and carrier phase 0, as described next. 

To remove the dependence on 6 we average the likelihood function L(a,, 6, 7), but 
not its logarithm, over all possible values of 8 inside the range [0, 277]. Assuming that 6 is 
uniformly distributed inside this range, which is usually justifiable, we may write 

lar 


Laylak, T) R 0 L(g, 0, 7) fe(9) dé 


Sa cai Satnav \ao 
Zr Jo exp Noo > T 


The exponent in L(a,, 0, T) is expressed by (see Problem 6.49) 





= xElnislae, 8,2) = = Relat de 


0 0 
2VE 
SENG Re[|a,%,(7)| exp( j(argl%,(7)] — arglaz] — 0))] (6.273) 
2VE 
= N |a,X,.(7)| cos(arg[%,(z)] — arg[la,] — 0) 


Hence, 


1 2a 
Lalar, 7) = 1f ex 


1 2a—arg|%,(7)|+arglaz] GE 
xp 





2VE 
N |a,%,(7)| cos(arg[%,(7)] — argia] — a) dé 
: (6.274) 


N | a,%,,(7)| cos (¢) ie) 


where, in the last line, we have made the substitution 


Qa J —argle,(7)|+arglaz] 


= arg|x,(t)] — argla,] — @ 
We now invoke the definition of the modified Bessel function of zero order, as shown by 
(see Appendix 3) 
1 Lar 


I(x) ese do (6.275) 


~ Tr ho 
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Hence, we may express the average likelihood function L,,(a,, T) as 


2VE, | 
Lalar, T) = wf No |a,%,(7)| (6.276) 





where %,(7) is the complex envelope of the matched filter output in the receiver due to the 
kth transmitted symbol a,. For M-ary PSK, we have 


|a,| = 1 for all k 


Hence, Equation (6.276) reduces to 


2VE 
Lav(ag, 7) = 1 EAG) I) (6.277) 


We thus see that averaging the likelihood function over the carrier phase 6 has also re. 
moved dependence on the transmitted symbol a, for M-ary PSK. 

Finally, taking account of the transmission of Lo independent symbols ao, a4, . . ; 
ar,-1ı we may express the overall likelihood function of 7 as 


Lg-1 
Il Lila; 7) 
k=0 
Lo-1 

2VE 
IT ol 


k=0 


Lal 7) 


(6.278) 





|%,(7) I) 


Now we can take the logarithm of La(7T) to obtain the log-likelihood function of 7 as 


Ine(t) = log Lavt?) 
Lo—1 2VE (6.279) 
> log i N itn] 


k=0 o 


0 





: To proceed further, we need to approximate /,,(7). To that end we first note that the 
modified Bessel function I(x) may be expanded in a power series as (see Appendix 3) 


(: 2m 
7x 
me 
hx) = 2 E 


For small values of x we may thus approximate Io{x} as 


2 

x 
I(x) = 1 += 
o(X) 4 


We may further simplify matters by using the approximation 


yA 
log h(x) = log| 1 + Py 
x2 
4 


For the problem at hand, small x corresponds to small signal-to-noise ratio, Under this 
condition, we may approximate Equation (6.279) as 


for small x 


Erat 
llth = De lalol? (6.280) 
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where, as mentioned earlier, £,(7) is the complex envelope of the matched filter output 
due to the kth transmitted symbol. 
Differentiating /,,(7) with respect to the group delay 7, we obtain 


al,,,(7) tat 
OT k TAF 7 
Etg (6.281) 
PA Re[% i ()F4(7)] 


N3 


where £ } (7T) is the complex conjugate of ž, (7) and 24 (7) is its derivative with respect to 7. 
Accordingly, we may define the error signal for timing recovery as (accounting for the 
scaling factor 2E/N3 in what follows) 

ela] = Re[X;,(7)£.(7)] 


where we have used 7 in place of k to be consistent with the notation in Figure 6.62. Let 
#, denote the estimate of the unknown delay 7 at time t = nT. Then, introducing the 
definitions 


X,(T) = X(nT + $p) 
and 
X(T) = &'(nT + 7,) 
we may reformulate the error signal e(z) as 
eln] = Re[&*(nT + #,)%/(nT + 4,)] (6.282) 
Calculation of the error signal e[n] requires the use of two filters: 


1. The complex matched filter for generating <,,(7). 

2. The derivative matched filter for generating X/,(7). 
The receiver is already equipped with the first filter. The second one is new. In practice, 
the additional computational complexity due to the derivative matched filter is objection- 
able. We may dispense with the need for it by using a finite difference to approximate the 
derivative X/(7) as 


1 
x'(nT + 4) = T [a(»r + La tenue) = a( wr = i + tsa) (6.283) 


where 7,,«12 are the timing estimates computed at nT + T/2. It is desirable to make one 
further modification to account for the fact that timing estimates are updated at multiples 
of the symbol period T and the only available quantities are 7,. Consequently, we replace 
+12 by 7, (which represents the latest estimate of 7) and replace 7-1). by %—1 (which is 
the estimate of 7 before the last one). We may thus rewrite Equation (6.283) as 


eee pte cles ee ae 
ZnT + 4%) = T EG + 7 + i) a( wr 7 + ir) (6.284) 


and so finally redefine the error signal as 


ejn] = Ref er(ur + ial a(wr + Za i) = a( nr = > + iall} (6.285) 


where the scaling factor 1/T is also accounted for in what follows. 
We are now ready to formulate the recursive algorithm for timing recovery: 


cin + 1] = cja] + yel[r] (6.286) 
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FIGURE 6.64 Nondata-aided early-late delay synchronizer. 


where y is the step-size parameter in which 2E/N& and 1/T are absorbed, and the error 
signal e[7] is defined by Equation (6.285). The c[] is a real number employed as the control 
for the frequency of an oscillator, referred to as a number-controlled oscillator (NCO). 
The scheme for implementing the timing recovery algorithm of Equations (6.285) and 
(6.286) is shown in Figure 6.64. This scheme is analogous to the continuous-time version 
of the early-late gate synchronizer widely used for timing recovery. It is thus referred to 
as a nondata-aided early-late delay (NDA-ELD) synchronizer. At every iteration, it works 


; £ T 
on three successive samples of the matched filter output, namely, &| nT + + #,|, 


T 
a( wr + | and a( wr + 37 ĉ,—1 |. The first sample is early and the last one is late, 


both with respect to the middle one. 

Note that we could have simplified the derivations presented in this section by using 
the band-pass to complex low-pass transformation described in Appendix 2. We did not 
do so merely for the sake of simplifying the understanding of the material presented here. 


6.15 Computer Experiments: 
Carrier Recovery and Symbol Timing 








In this section we illustrate the operations of the recursive Costas loop and nondata-aided 
early-late delay synchronizer by considering a coherent QPSK system with the following 
specifications: 
(i) Channel response: raised cosine (Nyquist) with rolloff factor a = 0.5. 
(ii) Loop filter: first-order digital filter with its transfer function defined by 
1 
B= = 9A) 
where yis the step-size parameter and A is a parameter to be defined. 
(iii) Loop bandwidth, Bz = 2% of the symbol rate 1/T; that is, BpT = 0.02. 
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Experiment 1: Carrier Phase Recovery 


In order to investigate the phase-acquisition behavior of the recursive Costas loop, we need 
to have the so-called S-curve of the phase-error generator. This is defined as the expectation 
of the adjustment signal e[n], conditioned on a fixed value of the phase error 


g=6- ô 
where @ is the actual value of the carrier phase and Ê is its estimate. That is, 
Sle) = Elen] | e] (6.288) 


Experimentally, S(p) is measured by opening the recursive Costas loop of Figure 6.62 and 
measuring the average of the adjustment signal e[], as indicated in Figure 6.65. 

The implementation procedure consists of the following steps. First, the complex 
envelope of the received signal is generated, which is given by 


x(t) = [OE epitaf- adet- %) + we) (6.289) 





where a, = 0, 7/2, 77, 37/4; 7, is the carrier delay and 7, is the group delay; and (t) is 
the complex-valued channel noise. The overall channel response g(t) is given by the Ny- 
quist pulse (see Section 4.5) 


_ Sin(wt/T) _cos(zat/T) 
800 = CIT) 1 — 4a IT? (6:230) 


where a = 0.5. As pointed out earlier, we assume that the symbol timing (i.e., group 
delay 7,) is known, and the problem is to estimate the carrier phase 6 = —27f,7,. The 
effect of @is to shift an element of the signal constellation in the manner indicated in Figure 
6.66. 

Using the experimental procedure described in Figure 6.65, the S-curve of the QPSK 
system may now be measured. Figure 6.674 shows the ideal S-curve, assuming an infinitely 
large signal-to-noise ratio. This curve displays discontinuities at p = +7/4, where 
m = 0, 1, 3,..., because of ambiguity encountered in the detection of the transmitted 


exp(-j4) 








Si) 


FIGURE 6.65 Scheme for measuring the S-curve for carrier phase recovery. 
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FIGURE 6.66 Illustrating the effect of carrier phase 8 on a state of the QPSK signal. 


symbol a,. The presence of channel noise tends to roundoff the discontinuities, as shown 
in the experimentally measured S-curve of Figure 6.676. The results presented in Figure 
6.67b were obtained for E/N, = 10 dB. Recall that the in-phase and quadrature compo- 
nents of the narrowband noise have an identical Gaussian distribution with zero mean 
and the same variance as the original narrowband noise; these two components define 


w(t). 
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FIGURE 6.67 Performance of recursive Costas loop. (a) S-curve for (E/No) = %. (b) S-curve for 


(E/N) = 10 dB. 
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FIGURE 6.68 Effects of varying the step-size parameter on convergence behavior of the recur- 
sive Costas loop. 


When steady-state conditions have been established, the estimated phase Ê will fluc- 
tuate around the true value 8. The extent of these fluctuations depends on the step-size 
parameter and the received signal-to-noise ratio: 


(i) Figure 6.68 plots the phase error ¢ versus the normalized time ¢/T for two different 
values of step-size parameter y, namely, 0.1 and 0.5, and fixed E/Ny = 20 dB. This 
figure clearly shows that the smaller we make y the smaller the steady-state fluctu- 
ations in the phase error ¢ will be. However, this improvement is attained at the 
expense of a slower rate of convergence of the algorithm. The number of iterations 
needed by the algorithm to reach steady-state is approximately given by 


1 
byte (6.291) 
The normalized bandwidth B,T is itself approximately given by 
ByT = x (6.292) 


where A is the slope of the S-curve measured at the origin. For y = 0.1, and 
BLT = 0.02, Equation (6.291) yields Lọ = 25 iterations, which checks with the solid 
curve plotted in Figure 6.68. Moreover, from Equations (6.291) and (6.292) we see 
that Lo is inversely proportional to y, which again checks with the results presented 
in Figure 6.68. 

(ii) Figure 6.69 plots the phase error ø versus the normalized time ¢/T for three different 
values of E/No, namely, 5, 10, and 30 dB, and fixed y = 0.08. We now see that the 
larger we make the signal-to-noise ratio, the smaller the steady-state fluctuations in 
the phase error œ will be. Moreover, the rate of convergence of the algorithm also 
improves with increased signal-to-noise ratio, which is intuitively satisfying. 
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FIGURE 6.69 Convergence behavior of the recursive Costas loop for varying E/No. 


Figure 6.70 plots the variance of the phase error (averaged over 100 trials of the 
experiment) versus E/No (measured in decibels) for BT = 0.02 and y = 0.08. This figure 
also includes a plot of the modified Cramér-Rao bound defined by** 


1 
2Lo(E/No) 


This bound is a modification of the ordinary Cramér-Rao bound, which is a lower bound 
on the variance of any unbiased estimator. The modification to this bound is made to 
overcome computational difficulties encountered in practical synchronization problems. 


MCRB(6) = (6.293) 
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FIGURE 6.70 Comparison of the measured tracking-error variance of the recursive Costas loop 
against theory for varying E/No. 
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FIGURE 6.71 Scheme for measuring the S-curve for the recursive early-late-delay synchronizer. 


In any event, the experimental and theoretical results presented in Figure 6.70 are in very 
close agreement for (E/No) = 10 dB. 


Experiment 2: Symbol Timing Recovery 


To measure the S-curve for the nondata-aided early-late delay synchronizer for symbol 
timing recovery, we may use the experimental set-up shown ion Figure 6.71, where the 5 
in S(6) refers to the timing offset. The S-curve so measured is plotted in Figure 6.72 for 
E/N, = 10 dB and E/Ny = œ. 

Figure 6.73 plots the normalized value of the experimentally measured symbol timing 
error versus E/Ny for two different values of step-size parameter y, namely, T/20 and 
1/200. This figure also includes theoretical plots of the corresponding modified Cramér- 
Rao bound of Equation (6.293) adapted for symbol-timing error. From the results pre- 
sented here, we observe that as the step-size parameter y is reduced, the normalized timing 
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FIGURE 6.72 S-curve of NDA-ELD synchronizer measured under noiseless and noisy 
conditions. 
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FIGURE 6.73 Comparison of tracking-error variance of the NDA-ELD synchronizer against 
theory for varying E/No and two step-size parameters. 


error is reduced and the range of E/No for which the modified Cramér-Rao bound holds 
(albeit in an approximate fashion) is enlarged. 


| 6.1 6 Summary and Discussion 


With the basic background theory on optimum receivers of Chapter 5 at our disposal, in 
this chapter we derived formulas for, or bounds on, the bit error rate for some important 
digital modulation techniques in an AWGN channel: 


1. Phase-shift keying (PSK), represented by 
» Coherent binary phase-shift keying (BPSK). 
> Coherent quadriphase shift keying (QPSK) and its variants, namely, the offset 

QPSK and 77/4-shifted QPSK. 

» Coherent M-ary PSK, which includes BPSK and QPSK as special cases with M =2 
and M = 4, respectively. Coherent M-ary PSK is used in digital satellite 
communications. 

> Differential phase-shift keying (DPSK), which may be viewed as the pseudo-non 
coherent form of PSK. 

2. Coherent M-ary quadrature amplitude modulation (QAM), which is a hybrid form 
of modulation that combines amplitude and phase-shift keying. For M = 4 it includes 
QPSK as a special case. M-ary QAM is basic to the construction of high-speed 
voiceband modems. 

3. Frequency-shift keying (FSK), represented by 
> Coherent binary frequency-shift keying. 
>. Coherent forms of minimum shift keying (MSK) and Gaussian minimum shift 

keying (GMSK); GMSK is basic to the construction of GSM wireless 
communications. 
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= Coherent M-ary FSK. 
= Noncoherent binary FSK. 


In this chapter we also studied two alternative techniques for passband data trans- 
mission: carrierless amplitude/phase modulation (CAP) and discrete multitone (DMT). In 
the case of an AWGN channel, the performance of CAP and DMT are equivalent because 
the DMT may be viewed as a linear reversible transformation of a single-carrier modulated 
signal. However, they perform quite differently in a practical setting that deviates from 
this idealized model.** DMT has been standardized for use on asymmetric digital sub- 
scriber lines (ADSLs) using twisted pairs. CAP, with the use of decision feedback equali- 
zation, provides another approach for solving the ADSL problem. CAP is also used for 
data transmission in local area networks for premises’ distribution systems. 

DMT is a form of multichannel modulation, and so is orthogonal frequency-division 
multiplexing (OFDM). The basic difference between DMT and OFDM is that DMT per- 
mits the use of loading to optimize information capacity, whereas OFDM does not. This 
difference arises because of their different domains of application. DMT applies to two- 
wire channels such as ADSLs, whereas OFDM applies to broadcasting and wireless 
channels. 

Irrespective of the digital modulation system of interest, synchronization of the re- 
ceiver to the transmitter is essential to the operation of the system. Symbol timing recovery 
is required whether the receiver is coherent or not. If the receiver is coherent, we also 
require provision for carrier recovery. In the latter part of the chapter, we discussed non- 
data-aided synchronizers to cater to these two requirements with emphasis on M-ary 
phase-shift keying signals in which the carrier is suppressed. The presentation focused on 
iterative synchronization techniques that are naturally suited for the use of digital signal 
processing. 


| NOTES AND REFERENCES 


1. For an early tutorial paper reviewing different digital modulation techniques (ASK, FSK, 
and PSK) based on a geometric viewpoint, see Arthurs and Dym, (1962). See also the 
following list of books: 


> Anderson (1998, Chapter 3) 

% Benedetto and Biglieri (1999, Chapters 4 and 5) 
P Lee and Messerschmitt (1994, Part II) 

> Proakis (1995, Chapter 5) 

> Sklar (1988, Chapter 3) 

> Viterbi and Omura (1979, pp. 47-127) 


2. For an early paper on the offset QPSK, see Gitlin and Ho (1975). 


3. The a/4-shifted QPSK was first described in the open literature in Akaiwa and Negata 
(1987). 


4. Chennakeshu and Sauliner (1993) use computer simulations to study the performance of 
m/4-shifted QPSK in a digital wireless communications environment, The pulse-shaping 
signal used in the generation of the 7/4-shifted QPSK signal is based on the square root 
raised cosine spectrum (see Problem 4.38). In this latter paper, it is shown that the perfor- 
mance of 77/4-shifted QPSK may degrade rapidly in such an environment. The differential 
detector of Figure 6.13 follows Chennakeshu and Sauliner (1993). 
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5. 
6. 


10. 


11. 


12. 


13. 


14. 


For a derivation of Equation (6.65), see Cioffi (1998). 


The derivation of Equation (6.76) was first reported in a 1975 internal Bell Laborato 
memorandum authored by Werner. A little later, Falconer (1975) issued another Bel] Lah 
oratories memorandum, in which it was pointed out the symbol rotation is not oe 
needed if we did not want to be compatible with existing QAM or other bandpass disia 
thereby simplifying the mathematical representation of CAP signals (and therefore ae 
implementation), as shown in Equation (6.77). However, the terminology “CAP” wag as 
coined until 1987 when carrierless amplitude/phase modulation was replaced by Cap i 
the standards representative, Garry Smith, of Bell Laboratories. The first detailed discussion 
of CAP in the context of digital subscriber lines was presented in a two-part report h 
Werner (1992, 1993). In‘a separate report by Chen, Im, and Werner (1992), the feasibiliy 
of CAP for use on digital subscriber lines was studied; see also the book by Chen (1998) 
pp. 461-473. The application of CAP to local area networks, involving the use of twisted 
pairs for lengths less than 100 m, is discussed in the paper by Im and Werner (1995); the 
maximum length of 100 m is specified by a standard for the wiring of premises. 

The digital implementation of a baseband equalizer similar to the CAP receiver of 
Figure 6.24 is discussed in Mueller and Werner (1982). 


. The MSK signal was first described in Doelz and Heald (1961). For a tutorial review of 


MSK and comparison with QPSK, see Pasupathy (1979). Since the frequency spacing is 
only half as much as the conventional spacing of 1/T, that is used in the coherent detection 
of binary FSK signals, this signaling scheme is also referred to as fast FSK; see deBuda 
(1972). 


. For early discussions of Gaussian MSK, see Murota and Hirade (1981) and Ishizuka and 


Hirade (1980). 


. The analytical specification of the power spectral density of digital FM is difficult to han- 


dle, except for the case of a rectangular shaped modulating pulse. The paper by Garrison 
(1975) presents a procedure based on the selection of an appropriate duration-limited/ 
level-quantized approximation for the modulating pulse. The equations developed therein 
are particularly suitable for machine computation of the power spectra of digital FM sig- 
nals; see the book by Stiiber (1996). 


A detailed analysis of the spectra of M-ary FSK for an arbitrary value of frequency deviation 
is presented in the paper by Anderson and Salz (1965). The results shown plotted in Figure 
6.36 represent a special case of a formula derived in that paper for a frequency deviation 
of k= 0.5. 


The standard method of deriving the bit error rate for noncoherent binary FSK, presented 
in McDonough and Whalen (1995) and that for differential phase-shift keying presented 
in Arthurs and Dym (1962), involves the use of the Rician distribution. This distribution 
arises when the envelope of a sine wave plus additive Gaussian noise is of interest; see 
Chapter 1 for a discussion of the Rician distribution. The derivations presented in Section 
6.6 avoid the complications encountered in the standard method. 


The optimum receiver for differential phase-shift keying is discussed in Simon and Divsalar 
(1992). 


For a technical discussion of various kinds of modems, with emphasis on their operational 
characteristics, see the books of Lewart (1988) and Hold (1997). 


In a two-part paper by Wei (1984), differential encoding is applied to convolutional chan- 
nel coding. Several eight-state convolutional encoders are described therein, which result 
in codes that are transparent to signal element rotations. In particular, in part I of the 
paper, Wei describes design rules and procedures for a 90-degree rotationally invariant 
convolutional code that has been adopted for use in the V.32 modem with trellis coding 


15. 


16. 


17. 


18. 


19. 


20. 
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Nonuniform sampling of band-limited signals is discussed in the paper by Yen (1956). The 
main results derived in that paper are contained in four generalized theorems. Equation 
(6.188) is based on Theorem III of Yen’s paper. 

In the paper by Kalet, Mazo, and Saltzberg (1993), this particular theorem due to 
Yen is used to formulate the fundamental philosophy underlying the design of the bidirec- 
tional digital modem; see also the patent by Ayanoglu et al. (1995). 


For a discussion of the second realization of a digital modem, see the article by Humblet 
and Troulis (1996). - 


For a detailed description of the V.34 high-speed modem standard, see Forney et al. (1996). 
The trellis codes used in the V.34 modem are due to Wei (1984, 1987). 


The idea of multichannel modulation may be traced to the early work of Chang (1966), 
Saltzberg (1967), and Weinstein and Ebert (1971). A mathematical treatment of the opti- 
mality of multitone modulation for a linear channel with severe intersymbol interference 
is presented in Kalet (1989). However, it was the work done by Cioffi and co-workers that 
led to the standardization of discrete multitone (DMT) for asymmetric digital subscriher 
lines; for details, see Ruiz et al. (1992), Chow and Cioffi (1995), Section 7.2 of the book 
by Starr et al. (1999), and Chapter 11 of Cioffi (1998). Problem 6.44 is adapted from Cioffi 
(1998). 


The method of Lagrange multipliers for determining the extreme values of the function 


y = f(x) 


subject to the constraint 


g(x) = 0 
follows from the following theorem: A necessary and sufficient condition for an extremum 
of a continuously differentiable function f(x) is that its differential with respect to x van- 
ishes at the critical (i.e., maximum and minimum) points of the function. Accordingly, at 
the critical points of f(x) we have 


of 

ae dx = 0 (1) 
Moreover, since p(x) = 0, its differential also vanishes as shown by 

ðe a 

ax dx = (2) 


Hence multiplying (2) by some parameter A and then adding the result to (1), we get 


a ð 

(2 FA že) d&æ=0 
Ox Ox 

Since dx is an independent increment, we immediately deduce that 


= (fla) +A oix) = 0 
Ix 


This equation is a mathematical statement of the method of Lagrange multipliers. The 
parameter A is called the Lagrange multiplier. The material presented in this note follows 
Sokolnikoff and Redheffer (1966, pp. 341-344). 


In the discrete Fourier transform (DFT), both the input and the output consist of sequences 
of numbers defined at uniformly spaced points in time and frequency, respectively. This 
feature makes the DFT ideally suited for numerical computation using the fast Fourier 
transform (FFT) algorithm. FFT algorithms are efficient because they use a greatly reduced 
number of arithmetic operations compared to the brute-force computation of the DFT. 
Basically, an FFT algorithm attains its computational efficiency by following a “divide and 
conquer” strategy, whereby the original DFT computation is decomposed successively into 
smaller DFT computations. For the case of an N-point DFT and N = 2}, the FFT algorithm 
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21. 


22. 


23. 


24, 


25. 


| PROBLEMS 


requires L = log,N stages of computation, with each stage of the computation involyin, 
complex multiplications and additions of order N. For detailed discussion of the FFT al 
gorithm, see Oppenheim and Schafer (1989, Chapter 9). 5i 


An overview of very-high-rate digital subscriber lines (VDSL) is presented in the Paper h 
Cioffi et al. (1999); this paper also includes a comparative discussion on VDSLs an 
Voiceband modems. 


For discussion of OFDM and its applications, see Casas and Leung (1991), LeFloch et aj 
(1989), and Zou and Wu (1995). For tutorial notes on OFDM and an extensive list of 
references, see Cimini and Li (1999). 


For detailed descriptions of phase recovery and symbol-timing recovery using classical 
synchronization systems, see Stiffler (1971), Lindsey (1972), and Lindsey and Simon (1973 
Chapters 2 and 9). : 

For a modern treatment of synchronization systems with emphasis on the use of 
discrete-time signal processing algorithms, see Mengali and D’Andrea (1997), Meyr, 
Moeneclaey, and Fechtel (1998). : d 


Equation (6.293) on the modified Cramér-Rao bound for phase recovery is derived in 
Mengali and D’Andrea (1997). 


Saltzberg (1998) discusses how the performances of CAP and DMT are affected by channel 
impairments and system imperfections in the context of ADSL application. The impair. 
ments/imperfections considered therein include impulse noise, narrowband interference 
(e.g., RF ingress from am over-the-air AM radio transmission), timing jitter caused by 
imperfect synchronization, and system nonlinearities. 


Amplitude-Shift Keying 
6.1 In the on-off keying version of an ASK system, symbol 1 is represented by transmitting a 


sinusoidal carrier of amplitude V2E,/T,, where E; is the signal energy per bit and T, is 
the bit duration. Symbol 0 is represented by switching off the carrier. Assume that symbols 
1 and 0 occur with equal probability. 
For an AWGN channel, determine the average probability of error for this ASK 

system under the following scenarios: 
(a) Coherent reception. 
(b). Noncoherent reception, operating with a large value of bit energy-to-noise spectral 

density ratio E,/No. 
Note: When x is large, the modified Bessel function of the first kind of zero order may be 
approximated as follows (see Appendix 3): 


exp(x 


Iy(x) = 





j 


Phase-Shift Keying 
6.2 A PSK signal is applied to a correlator supplied with a phase reference that lies within ¢ 


radians of the exact carrier phase. Determine the effect of the phase error gon the average 
probability of error of the system. 


6.3 Consider a phase-locked loop consisting of a multiplier, loop filter, and voltage-controlled 


oscillator (VCO). Let the signal applied to the multiplier input be a PSK signal defined by 
s(t) = A, cos[2af,t + k,mit)] 


6.4 


6.5 


6.6 


6.7 


6.8 


6.9 
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where k, is the phase sensitivity, and the data signal m(t) takes on the value +1 for binary 
symbol 1 and —1 for binary symbol 0. The VCO output is 


r(t) = A, sin[2mf.t + 0(2)] 


(a) Evaluate the loop filter output, assuming that this filter removes only modulated com- 
ponents with carrier frequency 2f.. 

(b) Show that this output is proportional to the data signal m(t) when the loop is phase 
locked, that is, 8(ż) = 0. - 

The signal component of a coherent PSK system is defined by 


s(t) = A.k sin(2mf.t) + AN] — K cos(2rf.t) 


where 0 = t = Ts, and the plus sign corresponds to symbol 1 and the minus sign corre- 

sponds to symbol 0. The first term represents a carrier component included for the purpose 

of synchronizing the receiver to the transmitter. 

(a) Draw a signal-space diagram for the scheme described here; what observations can 
you make about this diagram? 

(b) Show that, in the presence of additive white Gaussian noise of zero mean and power 
spectral density No/2, the average probability of error is 


E, = = AZT 
b eee. 


(c) Suppose that 10 percent of the transmitted signal power is allocated to the carrier 
component. Determine the E,/No required to realize a probability of error equal to 
107%. 

(d) Compare this value of E,/No with that required for a conventional PSK system with 
the same probability of error. 

(a) Given the input binary sequence 1100100010, sketch the waveforms of the in-phase 
and quadrature components of a modulated wave obtained by using the QPSK based 
on the signal set of Figure 6.6. 

(b) Sketch the QPSK waveform itself for the input binary sequence specified in part (a). 

Let P.r and P.o denote the probabilities of symbol error for the in-phase and quadrature 

channels of a narrowband digital communication system. Show that the average proba- 

bility of symbol error for the overall system is given by 


P, = Pur + Peg — ParPeo 


Equation (6.47) is an approximate formula for the average probability of symbol error 

for coherent M-ary PSK. This formula was derived using the union bound in light of the 

signal-space diagram of Figure 6.15. Given that message point 7, was transmitted, show 
that the approximate formula of Equation (6.47) may be derived directly from Figure 
6.15b. 

Find the power spectral density of an offset QPSK signal produced by a random binary 

sequence in which symbols 1 and 0 (represented by +1) are equally likely, and the symbols 

in different time slots are statistically independent and identically distributed. 

Vestigial sideband modulation (VSB), discussed in Chapter 2, offers another modulation 

method for passband data transmission. 

{a) In particular, a digital VSB transmission system may be viewed as a time-varying one- 
dimensional system operating at a rate of 2/T dimensions per second, where T is the 
symbol period. Justify the validity of this statement. 

(b) Show that digital VSB is indeed equivalent in performance to the offset QPSK. 
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6.10 The binary data stream 01101000 is applied to a 7/4-shifted DQPSK modulator that; 
initially in the state (¢1 = VE, $2 = 0) in Figure 6.112. Using the relationships betwe i 
input dibits and carrier-phase shifts summarized in Table 6.2, determine the phase ae 
occupied by the modulator in response to the specified data stream. 3 

6.11 Just as in an ordinary QPSK modulator, the output of a 7/4-shifted DQPSK modulato 
may be expressed in terms of its in-phase and quadrature components as follows: i 


s(t) = s,(t) cos(2af.t) — solt) sin(27f.2) 
Formulate the in-phase component s;(t) and quadrature component so(t) of the 1/4- 
shifted DQPSK signal. Hence, outline a scheme for the generation of 7/4-shifted DQPsK 
signals. 

6.12 An interesting property of 7/4-shifted DQPSK signals is that they can be demodulated 
using an FM discriminator. Demonstrate the validity of this property. The FM discrimi. 
nator is discussed in Chapter 2. 

6.13 Let Ad, denote the differentially encoded phase in the 7/4-shifted DQPSK. The symbol 
pairs (I, Q) generated by this scheme may be defined as 

I, = Ip cos(A@) — Qy-1 sin(AO,) 
Oy = Ir sin(A%) + Qg—1 cos(A@) 
where I, and Q, are the in-phase and quadrature components corresponding to the kth 
symbol. Show that this pair of relations can be expressed simply as 
I, = cos 0 
Qk = sin 0, 
where 0, is the absolute phase angle for the kth symbol. 


Quadrature-Amplitude Modulation 


6.14 Figure 6.53 shows a 240-QAM signal constellation, which may be viewed as an extended 

form of QAM cross constellation. 

(a) Identify the portion of Figure 6.53 that is a QAM square constellation. 

(b) Build on part (a) to identify the portion of Figure 6.53 that is a QAM cross 
constellation. 

(c) Hence, identify the portion of Figure 6.5 3 that is an extension to QAM cross 
constellation. 

6.15 Determine the transmission bandwidth reduction and average signal energy of 256-QAM, 
compared to 64-QAM. 

6.16 Two passband data transmission systems are to be compared. One system uses 16-PSK, 
and the other uses 16-QAM. Both systems are required to produce an average probability 
of symbol error equal to 10° 3. Compare the signal-to-noise ratio requirements of these 
two systems. 


Carrierless Amplitude/Phase Modulation (CAP) 


6.17 The two-dimensional CAP and M-ary QAM schemes are closely related. Do the following: 
(a) Given a QAM system, with a prescribed number of amplitude levels, derive the equiv- 
alent CAP system. 
(b) Perform the reverse of part (a). 
6.18 Show that the power spectral density of a CAP signal with a total of L amplitude levels 
is defined by 


2 
Sif) = FPA? 
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where | P(f)| is the magnitude spectrum of the passband in-phase pulse p(t); the o% is the 
variance of the complex symbols A; = a; + jb;, which is defined by 
L 


oi =F Dd (a + b) 


6.19 You are given the baseband raised-cosine spectrum G(f) pertaining to a certain rolloff 


factor a. Describe a frequency-domain procedure for evaluating the passband in-phase 
pulse p(t) and quadrature pulse p(t) that characterizé the corresponding CAP signal. 


Frequency-Shift Keying 


6.20 


6.21 


6.22 


The signal vectors s; and s, are used to represent binary symbols 1 and 0, respectively, in 
a coherent binary FSK system. The receiver decides in favor of symbol 1 when 


x's, > x's, 
where x's; is the inner product of the observation vector x and the signal vector s, where 
i = 1, 2. Show that this decision rule is equivalent to the condition x; > xz, where x, and 
Xz are the elements of the observation vector x. Assume that the signal vectors sı and s2 
have equal energy. 
An FSK system transmits binary data at the rate of 2.5 X 10° bits per second. During the 
course of transmission, white Gaussian noise of zero mean and power spectral density - 
10° 7° W/Hz is added to the signal. In the absence of noise, the amplitude of the received 
sinusoidal wave for digit 1 or 0 is 1 mV. Determine the average probability of symbol 
error for the following system configurations: 
(a) Coherent binary FSK 
(b) Coherent MSK 
(c) Noncoherent binary FSK 
(a) In a coherent FSK system, the signals s,(t) and s,(t) representing symbols 1 and 0, 
respectively, are defined by 
silt), s2(t) = A, col 2 £ £ sA 0stsT, 
Assuming that f, > Af, show that the correlation coefficient of the signals s(t) and s(t) 
is approximately given by 


Th 
I $1(t)so(t) dt 
p = =—— {= ginc(24fT;) 


Te 
Í silt) dt 
o 


(b) What is the minimum value of frequency shift Af for which the signals s,(t) and s(t) 
are orthogonal? 

(c) What is the value of Af that minimizes the average probability of symbol error? 

(d) For the value of Af obtained in part (c), determine the increase in E,/No required so 
that this coherent FSK system has the same noise performance as a coherent binary 
PSK system. 


6.23 A binary FSK signal with discontinuous phase is defined by 


[2E cosl 2a £ + af), + a| for symbol 1 
Te 2 

ieee col 2 £ a af), + J for symbol 0 
T; 2 


s(t) = 


472 


CHAPTER 6 = PASSBAND DATA TRANSMISSION 


where E, is the signal energy per bit, Ts is the bit duration, and @, and 6, are sam, i 
values of uniformly distributed random variables over the interval 0 to 27. In effect, T e 
two oscillators supplying the transmitted frequencies f; + A f!2 operate independent A 
each other. Assume that f, >> Af. Yo 
(a) Evaluate the power spectral density of the FSK signal. 
(b) Show that for frequencies far removed from the carrier frequency f., the power spec 
tral density falls off as the inverse square of frequency. j 
6.24 Set up a block diagram for the generation of Sunde’s FSK signal s(t) with continuous 
phase by using the representation given in Equation (6.104), which is reproduced here: 


s(t) = = cos( 22) cos(27f.t) + = sin( =) sin(27f.t) 
y te b y te b 


6.25 Discuss the similarities between MSK and offset QPSK, and the features that distinguish 
them. 

6.26 There are two ways of detecting an MSK signal. One way is to use a coherent receiver to 
take full account of the phase information content of the MSK signal. Another way is to 
use a noncoherent receiver and disregard the phase information. The second method offers 
the advantage of simplicity of implementation, at the expense of a degraded noise per. 
formance. By how many decibels do we have to increase the bit energy-to-noise density 
ratio E,/Np in the second case so as to realize an average probability of symbol error 
equal to 107% in both cases? 

6.27 (a) Sketch the waveforms of the in-phase and quadrature components of the MSK signal 

in response to the input binary sequence 1100100010. 
(b) Sketch the MSK waveform itself for the binary sequence specified in part (a). 

6.28 A nonreturn-to-zero data stream (of amplitude levels +1) is passed through a low-pass 
filter whose impulse response is defined by the Gaussian function 


Va z 
h(t) = = eo(-=F) 


where æ is a design parameter defined in terms of the filter’s 3-dB bandwidth by 


= flog2 1 
e [PS 


(a) Show that the transfer function of the filter is defined by 
H(f) = exp(-a7f?) 
Hence demonstrate that the 3-dB bandwidth of the filter is indeed equal to W. You 
may use Table A6.3 on Fourier-transform pairs. 
(b) Show that the response of the filter to a rectangular pulse of unit amplitude and 
duration T centered on the origin is defined by Equation (6.135). 
6.29 Plot the waveform of a GMSK modulator produced in response to the binary sequence 
1101000, assuming the use of a gain-bandwidth product WT, = 0.3. Compare your result 
with that of Example 6.5. 


6.30 Summarize the similarities and differences between the standard MSK and Gaussian- 
filtered MSK signals. 


Noncoherent Receivers 


6.31 In Section 6.8 we derived the formula for the bit error rate of noncoherent binary F SK 
as a special case of noncoherent orthogonal modulation. In this problem we revisit this 
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issue. As before, we assume that binary symbol 1 represented by signal s,(t) is transmitted. 

According to the material presented in Section 6.8, we note the following: 

> The random variable L, represented by the sample value /, of Equation (6.164) is Rayleigh 
distributed. 

> The random variable L, represented by the sample value /, of Equation (6.170) is Rician- 
distributed. 

The Rayleigh and Rician distributions are discussed in Chapter 1. Using the probability 

distributions defined in that chapter, derive the formula of Equation (6.181) for the BER 

of noncoherent binary FSK. 

6.32 Figure P6.32a shows a noncoherent receiver using a matched filter for the detection of a 
sinusoidal signal of known frequency but random phase, in the presence of additive white 
Gaussian noise. An alternative implementation of this receiver is its mechanization in the 
frequency domain as a spectrum analyzer receiver, as in Figure P6.32b, where the cor- 
relator computes the finite time autocorrelation function R,(7) defined by 


‘TT 
R7) = Í x(t)x(t + 7) dt, 0srsT 


Show that the square-law envelope detector output sampled at time t = T in Figure P6.324 
is twice the spectral output of the Fourier transformer sampled at frequency f = f, in 


Figure P6.325. 
Filter matched to Square-law x(t) A Output 
zl cos (2rf,,t); =| envelope a Se Output Correlator ceil sampled at 
OstsT detector Sample at fh, 


t=T 
(a) (b) 


FIGURE P6.32 


6.33 The binary sequence 1100100010 is applied to the DPSK transmitter of Figure 6.43a. 
(a) Sketch the resulting waveform at the transmitter output. 
(b) Applying this waveform to the DPSK receiver of Figure 6.436, show that, in the 
absence of noise, the original binary sequence is reconstructed at the receiver output. 
6.34 Differential M-ary PSK is the M-ary extension of binary DPSK. The present phase angle 
8, of the modulator at symbol time # is determined recursively by the relation 


2 
6, = O,-1 + (=) modulo 27 
where 6,1 is the previous phase angle and m, € {0, 1,..., M — 1} is the present mod- 
ulator input. The probability of symbol error for this M-ary modulation scheme is ap- 
proximately given by 


2E. {T 
P, = ete{ x, sin( 5), M24 


where it is assumed that E/Np is large. 

(a) Determine the factor by which the transmitted energy per symbol would have to be 
increased for the differential M-ary PSK to attain the same probability of symbol 
error as coherent M-ary PSK for M = 4. 

(b) For M = 4, by how many decibels is differential QPSK poorer in performance than 
coherent QPSK? 
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Comparison of Digital Modulation Schemes Using a Single Carrier 








6.35 Binary data are transmitted over a microwave link at the rate of 10° b/s, and the pow 
spectral density of the noise at the receiver input is 107° W/Hz. Find the average atni 
power required to maintain an average probability of error P, = 10~* for (a) coherent 
binary PSK, and (b) DPSK. 

6.36 The values of E,/No required to realize an average probability of symbol error P, = 19~4 
using coherent binary PSK and coherent FSK (conventional) systems are equal to 7.2 ang 
13.5, respectively. Using the approximation 

erfc(u) = l exp(—u°) 
Vru p 
determine the separation in the values of E,/No for P, = 1074, using 
(a) Coherent binary PSK and DPSK. 
(b) Coherent binary PSK and QPSK. 
(c) Coherent binary FSK (conventional) and noncoherent binary FSK. 
(d) Coherent binary FSK (conventional) and coherent MSK. 

6.37 In Section 6.10 we compared the noise performances of coherent binary PSK, coherent 
binary FSK, QPSK, MSK, DPSK, and noncoherent FSK by using the bit error rate as the 
basis of comparison. In this problem we take a different viewpoint and use the average 
probability of symbol error, Pe, to do the comparison. Plot P, versus E,/No for each of 
these schemes and comment on your results. 

6.38 The noise equivalent bandwidth of a bandpass signal is defined as the value of bandwidth 
that satisfies the relation 

2BS(f.) = P/2 
where 2B is the noise equivalent bandwidth centered around the midband frequency f, 
S(f,) is the maximum value of the power spectral density of the signal at f = fa and P is 
the average power of the signal. Show that the noise equivalent bandwidths of binary 
PSK, QPSK, and MSK are as follows: 
Type of Modulation Noise Bandwidth/Bit Rate 
Binary PSK 1.0 
QPSK 0.5 
MSK 0.62 
Note: You may use the definite integrals in Table A6.10. A discussion of noise equivalent 
bandwidth is presented in Appendix 2. 
Voiceband Modems 
6.39 (a) Refer to the differential encoder used in Figure 6.48a. Table 6.10 defines the phase 


changes induced in the V.32 modem by varying input dibits. Expand this table by 
including the corresponding previous and current values of the differential encoders 
output. Note that for every input dibit Q1 „Q27 there are four possible values for the 
differentially encoded dibit I,,,J2,, and likewise for its previous value Tip vay 

(b) The current quadbit applied to the V.32 modem with nonredundant coding is 0001. 
The previous output of the modem is 01. Find the code word output produced by the 
modem and its coordinates. 
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6.40 The V.32 modem standard with nonredundant coding uses a rectangular 16-QAM con- 
stellation. The model specifications are as follows: 
Carrier frequency = 1,800 Hz 
Symbol rate = 2,400 bands 
Data rate = 9,600 b/s 


Calculate (a) the average signal-to-noise ratio, and (b) the average probability of symbol 
error for this modem, assuming that E,,/No = 20dB. 


Multichannel Line Codes 


6.41 Consider the passband basis functions defined in Equation (6.196), where (t) is itself 
defined by Equation (6.197). Demonstrate the validity of Properties 1, 2, and 3 of these 
passband basis functions mentioned on pages 434 and 435. 

6.42 The water-filling solution for the loading problem is defined by Equation (6.213) subject 
to the constraint of Equation (6.210). Using this pair of relations, formulate a recursive 
algorithm for computing the allocation of the transmit power P among the N subchannels. 
The algorithm should start with (a) an initial total or sum noise-to-signal ratio 
NSR(?) = 0 for iteration i = 0, and (b) the subchannels sorted in terms of those with the 
smallest power allocation to the largest. 

6.43 The squared magnitude response of a linear channel, denoted by | H(f)|?, is shown in 
Figure P6.43. Assume that the gapT = 1 and the noise variance 02 = 1 for all subchannels. 
(a) Derive the formulas for the optimum powers P}, Pa, and P, allocated to the three 

subchannels of frequency bands (0, W1), (W1, W2), and (W>, W). 
(b) Given that the total transmit power P = 10, L = 2/3 and l, = 1/3, calculate the 
corresponding values of P}, P}, and P3. 


LHI? 


1 








FiGuneE P6.43 


6.44 Inthis problem we explore the use of singular value decomposition (SVD) as an alternative 
to the discrete Fourier transform for vector coding. This approach avoids the need for a 
cyclic prefix, with the channel matrix being formulated as 

hy by hi = h, 0 = 0 
0 bo h = b h, = 0 
Hae a i haat 


00 0 = by bh = by 
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6.45 


where the sequence ho, h1, . . - , b, denotes the sampled impulse response of the cha, 
The SVD of the matrix H is defined by ` 


H = UJA : Ox,]Vt 


nnel, 


where U is an N-by-N unitary matrix and V is an (N + »)-by-(N + ») unitary Matrix, 
that is, 3 


UUt =I 
wt =I 


where I is the identity matrix and the superscript + denotes Hermitian transposition, The 

A is an N-by-N diagonal matrix with singular values A, 1 = 1, 2,..., N. The Ox, is 

an N-by-v matrix of zeros. i 

(a) Using this decomposition, show that the N subchannels resulting from the use of 
vector coding are mathematically described by 


X, = AnAn + Wi 


The X, is an element of the matrix product Utx, where x is the received signal (channel 
output) vector. The A, is the nth symbol a,, + jb, and W,, is a random variable due 
to channel noise. 

(b) Show that the signal-to-noise ratio for vector coding as described herein is given by 


N* +p) 
(SNR )vector coding = r( i (: + See) Mp 
n=1 


where N* is the number of channels for each of which the allocated transmit power 
is nonnegative, (SNR), is the signal-to-noise ratio of subchannel 7, and T is a pre- 
scribed gap. 

(c) As the block length N approaches infinity, the singular values approach the magni- 
tudes of the channel Fourier transform. Using this result, comment on the relationship 
between vector coding and discrete multitone. 


Compare the performance of DMT and CAP with respect to the following channel 
impairments: 

(a) Impulse noise. 

(b) Narrowband interference. 


Assume that (1) the DMT has a large number of subchannels, and (2) the CAP systemis 
uncoded and its receiver uses a pair of adaptive filters for implementation. 


6.46 Orthogonal frequency-division multiplexing may be viewed as a generalization of M-ary 


FSK. Validate the rationale of this statement. 


Synchronization 


6.47 Figure P6.47 shows the block diagram of a continuous-time Mth power loop for phase 


recovery in an M-ary PSK receiver. 

(a) Show that the output of the Mth power-law device contains a tone of frequency Mf, 
where f. is the original carrier. 

(b) The oscillator in the phase-locked loop is set to a frequency equal to Mf. Justify this 
choice. 

(c) The Mth power loop suffers from a phase ambiguity problem in that it exhibits M 
phase ambiguities in the interval [0, 277]. Explain how this problem arises in the Mt 
power loop. How would you overcome the problem? 
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FIGURE P6.47 


6.48 (a) In the recursive algorithm of Equation (6.272) for phase recovery, the old estimate 
6[n] and the updated estimate [n + 1] of the carrier phase 0 are both measured in 
radians, Discuss the units in which the error signal e[n] and step-size parameter y are 
measured, 

(b) In the recursive algorithm of Equation (6.286) for symbol timing recovery, the control 
signals c[#] and c[z + 1] are both dimensionless. Discuss the units in which the error 
signal e[n] and step-size parameter y are measured. 


6.49 Using the definitions of Equations (6.264) and (6.265) for %, and a,, respectively, show 
that the exponent in the likelihood function L(a,, 0, 7) can be expressed as in Equation 
(6.273). 


6.50 In Section 6.14 we studied a non—data-aided scheme for carrier phase recovery, based on 
the log-likelihood function of Equation (6.260). In this problem we explore the use of 
this equation for data-aid carrier phase recovery. 


(a) Consider a receiver designed for a linear modulation system. Given that the receiver 
has knowledge of a preamble of length Lo, show that the maximum likelihood esti- 
mate of the carrier phase is defined by 


A ton} 
O= arel D arsi} 
É=0 
where the preamble {a,}£29' is a known sequence of complex symbols, and 
{%(&)}£2o" is the complex envelope of the corresponding received signal. 
(b) Using the result derived in part (a), construct a block diagram for the maximum 
likelihood phase estimator. 


478 


CHAPTER 6 © PASSBAND DATA TRANSMISSION 


Computer Experiment 


6.51 The purpose of this computer experiment is to compare the effect of a dispersive channel 


on the waveforms generated by the following passband modulation techniques: 
(a) Binary phase-shift keying (BPSK) 

(b) Quadriphase-shift keying (QPSK) 

(c) Minimum shift keying (MSK) 

(d) Gaussian MSK with time-bandwidth product WT, = 0.3 


The channel consists of a band-pass Butterworth filter of order 2N = 10 and 3-dB band- 
width 2B centered on the midband frequency f.. The low-pass equivalent of the channel 
has the squared magnitude response 


1 
HOP = pee 


The channel bandwidth is variable so as to illustrate its effect on the filtered modulated 
wave. 
Assuming the use of a coherent receiver, plot the waveforms of the modulated 

signals under (a), (b), (c) and (d) for the following channel bandwidths: 

(i) 2B = 12 kHz 

(ii) 2B = 16 kHz 
(iii) 2B = 20 kHz 
(iv) 2B = 24 kHz 

(v) 2B = 30 kHz 
Comment on your results. 


Hint. To perform the computations needed for this experiment, it is advisable to perform 
the computations in baseband by performing the band-pass to low-pass transformation 
described in Appendix 2. 





SPREAD-SPECTRUM 
| MODULATION 


This chapter introduces a modulation technique called spread-spectrum modulation, 
which is radically different from the modulation techniques that are covered in preceding 
chapters. In spread-spectrum modulation, channel bandwidth and transmit power are 
sacrificed for the sake of secure communications. 

Specifically, we cover the following topics: 


p> Spreading sequences in the form of pseudo-noise sequences, their properties, and methods 
of generation. 


> The basic notion of spread-spectrum modulation. 


> The two commonly used types of spread-spectrum modulation: direct sequence and 
frequency hopping. 


The material presented in this chapter is basic to wireless communications using code- 
division multiple access, which is covered in Chapter 8. 


| 7.1 Introduction 


A major issue of concern in the study of digital communications as considered in Chapters 
4, 5, and 6 is that of providing for the efficient use of bandwidth and power. Notwith- 
standing the importance of these two primary communication resources, there are situa- 
tions where it is necessary to sacrifice this efficiency in order to meet certain other design 
objectives. For example, the system may be required to provide a form of secure com- 
munication in a hostile environment such that the transmitted signal is not easily detected 
or recognized by unwanted listeners. This requirement is catered to by a class of signaling 
techniques known collectively as spread-spectrum modulation. 

The primary advantage of a spread-spectrum communication system is its ability to 
reject interference whether it be the unintentional interference by another user simulta- 
neously attempting to transmit through the channel, or the intentional interference by a 
hostile transmitter attempting to jam the transmission. 

The definition of spread-spectrum modulation’ may be stated in two parts: 


1. Spread spectrum is a means of transmission in which the data sequence occupies a 
bandwidth in excess of the minimum bandwidth necessary to send it. 


2. The spectrum spreading is accomplished before transmission through the use of a 
code that is independent of the data sequence. The same code is used in the receiver 
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(operating in synchronism with the transmitter) to despread the received signal so 
that the original data sequence may be recovered. 


Although standard modulation techniques such as frequency modulation and pulse-code 
modulation do satisfy part 1 of this definition, they are not spread-spectrum techniques 
because they do not satisfy part 2 of the definition. 

Spread-spectrum modulation was originally developed for military applications 
where resistance to jamming (interference) is of major concern. However, there are civilian 
applications that also benefit from the unique characteristics of spread-spectrum modu- 
lation. For example, it can be used to provide multipath rejection in a ground-based mobile 
radio environment. Yet another application is in multiple-access communications in which 
a number of independent users are required to share a common channel without an ex. 
ternal synchronizing mechanism; here, for example, we may mention a ground-based radio 
environment involving mobile vehicles that must communicate with a central station. More 
is said about this latter application in Chapter 8. 

In this chapter, we discuss principles of spread-spectrum modulation, with emphasis 
on direct-sequence and frequency-hopping techniques. In a direct-sequence spread- 
spectrum technique, two stages of modulation are used. First, the incoming data sequence 
is used to modulate a wideband code. This code transforms the narrowband data sequence 
into a noiselike wideband signal. The resulting wideband signal undergoes a second mod- 
ulation using a phase-shift keying technique. In a frequency-hop spread-spectrum tech- 
nique, on the other hand, the spectrum of a data-modulated carrier is widened by changing 
the carrier frequency in a pseudo-random manner. For their operation, both of these tech- 
niques rely on the availability of a noiselike spreading code called a pseudo-random or 
pseudo-noise sequence. Since such a sequence is basic to the operation of spread-spectrum 
modulation, it is logical that we begin our study by describing the generation and prop- 
erties of pseudo-noise sequences. 


| 7.2 Pseudo-Noise Sequences 


A pseudo-noise (PN) sequence is a periodic binary sequence with a noiselike waveform 
that is usually generated by means of a feedback shift register, a general block diagram of 
which is shown in Figure 7,1. A feedback shift register consists of an ordinary shift register 
made up of m flip-flops (two-state memory stages) and a logic circuit that are intercon- 
nected to form a multiloop feedback circuit. The flip-flops in the shift register are regulated 
by a single timing clock. At each pulse (tick) of the clock, the state of each flip-flop is 
shifted to the next one down the line. With each clock pulse the logic circuit computes a 


Output 
sequence 








Clock 


FIGURE 7.1 Feedback shift register. 
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Boolean function of the states of the flip-flops. The result is then fed back as the input to 
the first flip-flop, thereby preventing the shift register from emptying. The PN sequence so 
generated is determined by the length 77 of the shift register, its initial state, and the feed- 
back logic. 

Let s;(k) denote the state of the jth flip-flop after the kth clock pulse; this state may 
be represented by symbol 0 or 1. The state of the shift register after the kth clock pulse is 
then defined by the set {sı(k), s2(k), . . - , Sm(k)}, where k = 0. For the initial state, k is 
zero. From the definition of a shift register, we have 


k a k2=0 

slk + 1) = slk}, 1sjeom (7.1) 
where so(k) is the input applied to the first flip-flop after the kth clock pulse. According to 
the configuration described in Figure 7.1, so(k) is a Boolean function of the individual 
states sı(k), So(k),.-+5 Sm(k). For a specified length 7, this Boolean function uniquely 
determines the subsequent sequence of states and therefore the PN sequence produced at 
the output of the final flip-flop in the shift register. With a total number of m flip-flops, 
the number of possible states of the shift register is at most 2”. It follows therefore that the 
PN sequence generated by a feedback shift register must eventually become periodic with 
a period of at most 2”. 

A feedback shift register is said to be linear when the feedback logic consists entirely 
of modulo-2 adders. In such a case, the zero state (e.g., the state for which all the flip-flops 
are in state 0) is not permitted. We say so because for a zero state, the input so(k) produced 
by the feedback logic would be 0, the shift register would then continue to remain in the 
zero state, and the output would therefore consist entirely of 0s. Consequently, the period 
of a PN sequence produced by a linear feedback shift register with m flip-flops cannot 
exceed 2” — 1, When the period is exactly 2” — 1, the PN sequence is called a maximal- 
length-sequence or simply m-sequence. 


p EXAMPLE 7.1 


Consider the linear feedback shift register shown in Figure 7.2, involving three flip-flops. The 
input sọ applied to the first flip-flop is equal to the modulo-2 sum of s, and s3. It is assumed 
that the initial state of the shift register is 100 (reading the contents of the three flip-flops from 
left to right). Then, the succession of states will be as follows: 


100, 110, 111, 011, 101, 010, 001, 100,.... 


Modulo-2 
adder 









Fllp-flop 


2z |: 
sy 52 


So 
Clock os; ee | 


FIGURE 7.2 Maximal-length sequence generator for m = 3. 
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The output sequence (the last position of each state of the shift register) is therefore 
00111010... 


which repeats itself with period 2° — 1 = 7. 

Note that the choice of 100 as the initial state is arbitrary. Any of the other six permis. 
sible states could serve equally well as an initial state. The resulting output sequence would 
then simply experience a cyclic shift. 4 


u PROPERTIES OF MAXIMAL-LENGTH SEQUENCES” 


Maximal-length sequences have many of the properties possessed by a truly random 
binary sequence. A random binary sequence is a sequence in which the presence of binary 
symbol 1 or 0 is equally probable. Some properties of maximal-length sequences are as 
follows: 


1. In each period of a maximal-length sequence, the number of 1s is always one more than 
the number of Os. This property is called the balance property. 


2. Among the runs of 1s and of Os in each period of a maximal-length sequence, one- 
half the runs.of each kind are of length one, one-fourth are of length two, one-eighth are 
of length three, and so on as long as these fractions represent meaningful numbers of 
runs. This property is called the run property. By a “run” we mean a subsequehice 
of identical symbols (1s or 0s) within one period of the sequence. The length of this 
subsequence is the length of the run. For a maximal-length sequence generated by a 
linear feedback shift register of length m, the total number of runs is (N + 1)/2, where 
N=2"- 1, 


3. The autocorrelation function of a maximal-length sequence is periodic and binary- 
valued. This property is called the correlation property. 
The period of a maximum-length sequence is defined by 


N=2"-1 (7.2) 


_ where m is the length of the shift register. Let binary symbols 0 and 1 of the sequence be 


denoted by the levels —1 and +1, respectively. Let c(t) denote the resulting waveform of 
the maximal-length sequence, as illustrated in Figure 7.34 for N = 7. The period of the 
waveform c(t) is (based on terminology used in subsequent sections) 


T, = NT. (7.3) 


where T, is the duration assigned to symbol 1 or 0 in the maximal-length sequence. By 
definition, the autocorrelation function of a periodic signal c(t) of period T, is 


1 pte 
R(t) = T c(tìcit — 7) dt (74) 


-T;/2 
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Binary sequence O O 1 1 1 O 1 0 O 1 14 1 0 1 





+1 
af 2 








FIGURE 7.3 (a) Waveform of maximal-length sequence for length m = 3 or period N = 7. 
(b) Autocorrelation function. (c) Power spectral density. All three parts refer to the output of the 
feedback shift register of Figure 7.2. ` 


where the lag 7 lies in the interval (— T,/2,T,/2}); Equation (7.4) is a special case of Equation 
(1.26). Applying this formula to a maximal-length sequence represented by c(t), we get 
N+1 
1- =< 
Mele AST 


R(t) = 1 (7.5) 


a 
This result is plotted in Figure 7.36 for the case of m = 3 or N = 7. 





for the remainder of the period 
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From Fourier transform theory we know that periodicity in the time domain is trang. 
formed into uniform sampling in the frequency domain. This interplay between the time 
and frequency domains is borne out by the power spectral density of the maximal-length 
wave c(t). Specifically, taking the Fourier transform of Equation (7.5), we get the sampled 
spectrum 


1 1+N <= 
Sif) = seal + aS sne( 2) a $ x) (7.6 


n#0 








which is plotted in Figure 7.3¢ for m = 3 or N = 7. 

Comparing the results of Figure 7.3 for a maximal-length sequence with the corre. 
sponding results shown in Figure 1.11 for a random binary sequence, we may make the 
following observations: 


> For a period of the maximal-length sequence, the autocorrelation function R.(7) is 
somewhat similar to that of a random binary wave. 

» The waveforms of both sequences have the same envelope, sinc”(fT), for their power 
spectral densities. The fundamental difference between them is that whereas the ran- 
dom binary sequence has a continuous spectral density characteristic, the correspond- 
ing characteristic of a maximal-length sequence consists of delta functions spaced 
INT, Hz apart. 


As the shift-register length m, or equivalently, the period N of the maximal-length sequence 
is increased, the maximal-length sequence becomes increasingly similar to the random 
binary sequence. Indeed, in the limit, the two sequences become identical when N is made 
infinitely large. However, the price paid for making N large is an increasing storage require- 
ment, which imposes a practical limit on how large N can actually be made. 


CHOOSING A MAXIMAL-LENGTH SEQUENCE 


Now that we understand the properties of a maximal-length sequence and the fact that 
we can generate it using a linear feedback shift register, the key question that we need to 
address is: How do we find the feedback logic for a desired period N? The answer to this 


i TABLE 7.1 Maximal-length sequences of shift-register lengths 2—8 





Shift-Register 
Length, m Feedback Taps 

2* (2, 1] 
3* (3, 1] 
4 [4, 1] 
5* [5, 2], [5, 4, 3, 2], [5, 4, 2, 1] 
6 [6, 1], [6, 5, 2, 1], [6, 5, 3, 2] : 
us [7, 11, [7, 3], [7, 3, 2, 1], 17, 4, 3, 2], [7, 6, 4, 2], £7, 6, 3, 1], [7, 6, $, 2]; 


7, 6, 5, 4, 2, 1], [7, 5, 4, 3, 2, 1] 
8 [8, 4, 3, 2], [8, 6, 5, 3], [8, 6, 5, 2], [8, 5, 3, 1], [8, 6, 5, 1], (8, 7, 6, 11, 
[8, 7, 6, 5, 2, 1], [8, 6, 4, 3, 2, 1] 
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question is to be found in the theory of error-control codes, which is covered in Chapter 
10. The task of finding the required feedback logic is made particularly easy for us by 
virtue of the extensive tables of the necessary feedback connections for varying shift- 
register lengths that have been compiled in the literature. In Table 7.1, we present the sets 
of maximal (feedback) taps pertaining to shift-register lengths m = 2, 3,..., 8.7 Note 
that asm increases, the number of alternative schemes (codes) is enlarged. Also, for every 
set of feedback connections shown in this table, there is an “image” set that generates an 
identical maximal-length code, reversed in time sequence. 

The particular sets identified with an asterisk in Table 7.1 correspond to Mersenne 
prime length sequences, for which the period N is a prime number. 


P EXAMPLE 7.2 


Consider a maximal-length sequence requiring the use of a linear feedback-shift register of 
length m = 5. For feedback taps, we select the set [5, 2] from Table 7.1. The corresponding 
configuration of the code generator is shown in Figure 7.4a. Assuming that the initial state is 
10000, the evolution of one period of the maximal-length sequence generated by this scheme 
is shown in Table 7.2a, where we see that the generator returns to the initial 10000 after 31 
iterations; that is, the period is 31, which agrees with the value obtained from Equation (7.2). 
Suppose next we select another set of feedback taps from Table 7.1, namely, [5, 4, 2, 1]. 
The corresponding code generator is thus as shown in Figure 7.40. For the initial state 10000, 
we now find that the evolution of the maximal-length sequence is as shown in Table 7.25. 
Here again, the generator returns to the initial state 10000 after 31 iterations, and so it should. 
But the maximal-length sequence generated is different from that shown in Table 7.2a. 
Clearly, the code generator of Figure 7.44 has an advantage over that of Figure 7.46, 
as it requires fewer feedback connections. <q 
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FIGURE 7.4 Two different configurations of feedback shift register of length m = 5. (a) Feed- 
back connections [5, 2]. (b) Feedback connections [5, 4, 2, 1]. 
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| 7.3 A Notion of Spread Spectrum 











An important attribute of spread-spectrum modulation is that it can provide protection 
against externally generated interfering (jamming) signals with finite power. The jamming 
signal may consist of a fairly powerful broadband noise or multitone waveform that is 
directed at the receiver for the purpose of disrupting communications. Protection against 
jamming waveforms is provided by purposely making the information-bearing signal oc. 
cupy a bandwidth far in excess of the minimum bandwidth necessary to transmit it. This 
has the effect of making the transmitted signal assume a noiselike appearance so as to 
blend into the background. The transmitted signal is thus enabled to propagate through 
the channel undetected by anyone who may be listening. We may therefore think of spread 
spectrum as a method of “camouflaging” the information-bearing signal. 

One method of widening the bandwidth of an information-bearing (data) sequence 
involves the use of modulation. Let {b,} denote a binary data sequence, and {c,} denote a 
pseudo-noise (PN) sequence. Let the waveforms b(t) and c(t) denote their respective polar 
nonreturn-to-zero representations in terms of two levels equal in amplitude and opposite 
in polarity, namely, +1. We will refer to b(t) as the information-bearing (data) signal, and 
to c(t) as the PN signal. The desired modulation is achieved by applying the data signal 
b(t) and the PN signal c(t) to a product modulator or multiplier, as in Figure 7.54. We 
know from Fourier transform theory that multiplication of two signals produces a signal 
whose spectrum equals the convolution of the spectra of the two component signals. Thus, 
if the message signal b(t) is narrowband and the PN signal c(t) is wideband, the product 
(modulated) signal m(t) will have a spectrum that is nearly the same as the wideband PN 
signal. In other words, in the context of our present application, the PN sequence performs 
the role of a spreading code. 

By multiplying the information-bearing signal b(t) by the PN signal c(t), each infor- 
mation bit is “chopped” up into a number of small time increments, as illustrated in the 
waveforms of Figure 7.6. These small time increments are commonly referred to as chips.. 

For baseband transmission, the product signal m(t) represents the transmitted signal. 
We may thus express the transmitted signal as 


mlt) = c(t)b(t) (7.7) 


bl) m(t) m(t) r(t) 





c(t) it) 
(a) (b) 


z(t) J> v Decision m> Say lifv>0 


me o device 





> Say Oiv <0 








ole) Threshold = 0 


(o) 
FIGURE 7.5 Idealized model of baseband spread-spectrum system. (a) Transmitter. (b) Channel. 
(c) Receiver. 
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Figure 7.6 Illustrating the waveforms in the transmitter of Figure 7.5a. 


The received signal r(t) consists of the transmitted signal m(t) plus an additive interference 
denoted by i(¢), as shown in the channel model of Figure 7.56. Hence, 


r(t) = m(t) + i(t) 


= c(t)b(t) + i(t) (7.8) 


To recover the original message signal b(t), the received signal r(t) is applied to a 
demodulator that consists of a multiplier followed by an integrator, and a decision device, 
as in Figure 7.5c. The multiplier is supplied with a locally generated PN sequence that is 
an exact replica of that used in the transmitter. Moreover, we assume that the receiver 
operates in perfect synchronism with the transmitter, which means that the PN sequence 
in the receiver is lined up exactly with that in the transmitter. The multiplier output in the 
receiver is therefore given by 


z(t) = c(t)r(t) i 
= c*(t)b(t) + c(t)i(t) (7.9) 


Equation (7.9) shows that the data signa] b(t) is multiplied twice by the PN signal c(t), 
whereas the unwanted signal i(t) is multiplied only once. The PN signal c(t) alternates 
between the levels —1 and +1, and the alternation is destroyed when it is squared; hence, 


: e(t)=1 for all ¢ (7.10) 
Accordingly, we may simplify Equation (7.9) as 
z(t) = b(t) + e(#)i(t) (7.11) 
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We thus see from Equation (7.11) that the data signal b(t) is reproduced at the multiplier 
output in the receiver, except for the effect of the interference represented by the additive 
term c(t}i(t). Multiplication of the interference i(t) by the locally generated PN signal cit) 
means that the spreading code will affect the interference just as it did the original signal 
at the transmitter. We now observe that the data component b(t) is narrowband, whereas 
the spurious component ¢(t)i(t) is wideband. Hence, by applying the multiplier output to 
a baseband (low-pass) filter with a bandwidth just large enough to accommodate the 
recovery of the data signal b(t), most of the power in the spurious component c(t)i(t) is 
filtered out. The effect of the interference i(t) is thus significantly reduced at the receiver 
output. 

In the receiver shown in Figure 7.5c, the low-pass filtering action is actually per. 
formed by the integrator that evaluates the area under the signal produced at the multiplier 
output. The integration is carried out for the bit interval 0 = ¢ = T,, providing the sample 
value v. Finally, a decision is made by the receiver: If v is greater than the threshold of 
zero, the receiver says that binary symbol 1 of the original data sequence was sent in the 
interval 0 = t = Tp, and if v is less than zero, the receiver says that symbol 0 was sent; if 
v is exactly zero the receiver makes a random guess in favor of 1 or 0. 

In summary, the use of a spreading code (with pseudo-random properties) in the 
transmitter produces a wideband transmitted signal that appears noiselike to a receiver 
that has no knowledge of the spreading code. From the discussion presented in Section 
7.2, we recall that (for a prescribed data rate) the longer we make the period of the spread- 
ing code, the closer will the transmitted signal be to a truly random binary wave, and the 
harder it is to detect. Naturally, the price we have to pay for the improved protection 
against interference is increased transmission bandwidth, system complexity, and process- 
ing delay. However, when our primary concern is the security of transmission, these are 
not unreasonable costs to pay. i 


7.4 Direct-Sequence Spread Spectrum 
with Coherent Binary Phase-Shift Keying 


The spread-spectrum technique described in the previous section is referred to as direct- 
sequence spread spectrum. The discussion presented there was in the context of baseband 
transmission. To provide for the use of this technique in passband transmission over a 
satellite channel, for example, we may incorporate coherent binary phase-shift keying 
(PSK) into the transmitter and receiver, as shown in Figure 7.7. The transmitter of Figure 
7.7q first converts the incoming binary data sequence {b,} into a polar NRZ waveform 
b(t), which is followed by two stages of modulation. The first stage consists of a product 
modulator or multiplier with the data signal b(t) (representing a data sequence) and the 
PN signal c(t) (representing the PN sequence) as inputs. The second stage consists of a 
binary PSK modulator. The transmitted signal x(t) is thus a direct-sequence spread binary 
phase-shift-keyed (DS/BPSK) signal. The phase modulation 4(t) of x(t) has one of two 
values, 0 and 77, depending on the polarities of the message signal b(t) and PN signal c(t) 
at time t in accordance with the truth table of Table 7.3. 

Figure 7.8 illustrates the waveforms for the second stage of modulation. Part of the 
modulated waveform shown in Figure 7.6c is reproduced in Figure 7.8a; the waveform 
shown here corresponds to one period of the PN sequence. Figure 7.85 shows the wave 
form of a sinusoidal carrier, and Figure 7.8c shows the DS/BPSK waveform that results 
from the second stage of modulation. 
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FIGURE 7.7 Direct-sequence spread coherent phase-shift keying. (a) Transmitter. (b) Receiver. 


The receiver, shown in Figure 7.7b, consists of two stages of demodulation. In the 
first stage, the received signal y(t) and a locally generated carrier are applied to a product 
modulator followed by a low-pass filter whose bandwidth is equal to that of the original 
message signal 7(t). This stage of the demodulation process reverses the phase-shift keying 
applied to the transmitted signal. The second stage of demodulation performs spectrum 
despreading by multiplying the low-pass filter output by a locally generated replica of the 
PN signal c(t), followed by integration over a bit interval 0 = t = T}, and finally decision- 
making in the manner described in Section 7.3. 


u MODEL FOR ANALYSIS 


In the normal form of the transmitter, shown in Figure 7.7a, the spectrum spreading is 
performed prior to phase modulation. For the purpose of analysis, however, we find it 
more convenient to interchange the order of these operations, as shown in the model of 


TABLE 7.3 Truth table for phase modulation 
O(c), radians 








Polarity of Data 
Sequence b(t) at Time t 








Polarity of PN 
sequence C(t) at time t 
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FIGURE 7.8 (a) Product signal m(t) = c(t)b(t). (b) Sinusoidal carrier. (c) DS/BPSK signal. 


Figure 7.9. We are permitted to do this because the spectrum spreading and the binary 
phase-shift keying are both linear operations; likewise for the phase demodulation and 
spectrum despreading. But for the interchange of operations to be feasible, it is important 
to synchronize the incoming data sequence and the PN sequence. The model of Figure 7.9 
also includes representations of the channel and the receiver. In this model, it is assumed 
that the interference j(t) limits performance, so that the effect of channel noise may be 
ignored. Accordingly, the channel output is given by 


y(t) = x(t) + j(t) 
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FIGURE 7.9 Model of direct-sequence spread binary PSK system. 
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where s(t) is the binary PSK signal, and c(t) is the PN signal. In the channel model included 
in Figure 7.9, the interfering signal is denoted by j(t). This notation is chosen purposely 
to be different from that used for the interference in Figure 7.56. The channel model in 
Figure 7.9 is passband in spectral content, whereas that in Figure 7.50 is in baseband form. 

In the receiver, the received signal y(t) is first multiplied by the PN signal c(f) yielding 
an output that equals the coherent detector input u(t). Thus, 


ll 


u(t) = c(t)y(t) 

c*(t)s(t) + c(z)j(t) (7.13) 
= s(t) + c(t)j(t) 

In the last line of Equation (7.13), we have noted that, by design, the PN signal c(t) satisfies 

the property described in Equation (7.10), reproduced here for convenience: 


e(t)=1 for all t 


Equation (7.13) shows that the coherent detector input u(t) consists of a binary PSK signal 
s(t) embedded in additive code-modulated interference denoted by c(t)j(t). The modulated 
nature of the latter component forces the interference signal (jammer) to spread its spec- 
trum such that the detection of information bits at the receiver output is afforded increased 
reliability. 


E SYNCHRONIZATION 


For its proper operation, a spread-spectrum communication system requires that the locally 
generated PN sequence used in the receiver to despread the received signal be synchronized 
to the PN sequence used to spread the transmitted signal in the transmitter.* A solution 
to the synchronization problem consists of two parts: acquisition and tracking. In acqui- 
sition, or coarse synchronization, the two PN codes are aligned to within a fraction of the 
chip in as short a time as possible. Once the incoming PN code has been acquired, tracking, 
or fine synchronization, takes place. Typically, PN acquisition proceeds in two steps. First, 
the received signal is multiplied by a locally generated PN code to produce a measure of 
correlation between it and the PN code used in the transmitter. Next, an appropriate 
decision-rule and search strategy is used to process the measure of correlation so obtained 
to determine whether the two codes are in synchronism and what to do if they are not. As 
for tracking, it is accomplished using phase-lock techniques very similar to those used for 
the local generation of coherent carrier references. The principal difference between them 
lies in the way in which phase discrimination is implemented. 


7.5 Signal-Space Dimensionality 
and Processing Gain 


Having developed a conceptual understanding of spread-spectrum modulation and a 
method for its implementation, we are ready to undertake a detailed mathematical analysis 
of the technique. The approach we have in mind is based on the signal-space theoretic 
ideas of Chapter 5. In particular, we develop signal-space representations of the transmit- 
ted signal and the interfering signal (jammer). 
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In this context, consider the set of orthonormal basis functions: 


2 
dalt) = E cos(27f,t), kT. = ts {k + 1)T, (7.14) 
7 0, otherwise 
= b sin(27f.t), kT. = ts (k + 1)T, (7.15) 
0, i otherwise 


k=0,1,...,.N71 


where T, is the chip duration, and N is the number of chips per bit. Accordingly, we may 
describe the transmitted signal x(t) for the interval of an information bit as follows: 


x(t) = c(t)s(t) 


+ = c(t) cos(2mf:t) (7.16) 
b 


ll 
+ 


N-1 
= R D cedelt), á 0st=T% 
É=0 


where E, is the signal energy per bit; the plus sign corresponds to information bit 1, and 
the minus sign corresponds to information bit 0. The code sequence {cos C1, +++ Cn—i} 
denotes the PN sequence, with c, = +1. The transmitted signal x(t) is therefore N- 
dimensional in that it requires a minimum of N orthonormal functions for its 
representation. . 

Consider next the representation of the interfering signal (jammer), j(z). Ideally, the 
jammer likes to place all of its available energy in exactly the same N-dimensional signal 
space as the transmitted signal x(t); otherwise, part of its energy goes to waste. However, - 
the best that the jammer can hope to know is the transmitted signal bandwidth. Moreover, 
there is no way that the jammer can have knowledge of the signal phase. Accordingly, we 
may-represent the jammer by the general form 


N-1 N-1 Akg 
jt) = > kolt) + Diehl), OFt=T, (7.17) 
k=0 k=0 
where 
Ty 
ie = I, jlt)d(t) dt, k= 0,1,...,N—1 (7.18) 
and 
Tp x 
je= J, del) dte &=0,1,...,N—1 (7.19) 


Thus the interference (t) is 2N-dimensional; that is, it has twice the number of dimensions 
required for representing the transmitted DS/BPSK signal x(t). In terms of the represen- 
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tation given in Equation (7.17), we may express the average power of the interference j(t) 
as follows: 


Ty 


1 é 
Te P(t) dt 


b 0 

1 N-1 s 1 
=— Z + — 

Lea E 


N-1 (7.20) 
2 
2, fk 


Moreover, due to lack of knowledge of signal phase, the best strategy a jammer can apply 
is to place equal energy in the cosine and sine coordinates defined in Equations (7.18) and 
(7.19); hence, we may safely assume 


N=1 N-1 ia 
Dy k= > ik (7.21) 
=o KZO 
Correspondingly, we may simplify Equation (7.20) as 
2 N-1 
J=5 Dik (7.22) 
b k=0 


Our aim is to tie these results together by finding the signal-to-noise ratios measured at 
the input and output of the DS/BPSK receiver in Figure 7.9. To that end, we use Equation 
(7.13) to express the coherent detector output as 


a 


= V, + Ug 


where the components v, and v,; are due to the despread binary PSK signal, s(t), and the 
spread interference, c(z)j(t), respectively. These two components are defined as follows: 


Te 
v, = E [ s(t) cos(2af,t) dt (7.24) 
and 
2" 
vg = fe Í c(t)j(t) cos(2af:t) dt (7.25) 


Consider first the component v, due to the signal. The despread binary PSK signal 
s(t) equals 


s(t) = + FE cos(2r f:t), 0sts Te (7.26) 


where the plus sign corresponds to information bit 1, and the minus sign corresponds to 
information bit 0. Hence, assuming that the carrier frequency f; is an integer multiple of 
1/T,, we have 


v, = +VE, (7.27) 
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Consider next the component v; due to interference. Expressing the PN signal c(t} 
in the explicit form of a sequence, {co, ¢1,--- > cn_1}, we may rewrite Equation (7.25) in 
the corresponding form 


2 No} (R+T, 
vj = B 2 Ch iP j(t) cos(2arf.t} dt (7.28) 


Using Equation (7.14) for ,(¢), and then Equation (7.18) for the coefficient jz, we may 
redefine v,; as 


T No Ti, 
vg = [ae Dy | jelt) at 
Te é=0 0 


Ter 
= IT > Ckfk 


Te k=0 


(7.29) 


We next approximate the PN sequence as an independent and identically distributed (i.i.d.) 
binary sequence. We emphasize the implication of this approximation by recasting Equa- 
tion (7.29) in the form 


T N-1 
Vi = fat D, Cej (7.30) 
Th k=0 


where V; and C, are random variables with sample values v,; and c,, respectively. In 
Equation (7.30), the jammer is assumed to be fixed. With the C, treated as i.i.d. random 
variables, we find that the probability of the event C, = +1 is 


PIG, = 1) = (G = -1) = (7.31) 
Accordingly, the mean of the random variable Vy is zero since, for fixed k, we have 
E(Cyjs lja] = jxP(C, = 1) — jaP(Ck = —1) 
= the — Gk (7.32) 
=0 


For a fixed vector j, representing the set of coefficients jo; jis » + + > jn—1, the variance of Vaj 
is given by 


4 Na} 

var[Velil = 3 2s Ž (7.33) 
N é= 

Since the spread factor N = T,/T., we may use Equation (7.22) to express this variance in 
terms of the average interference power J as 


varlVglil = (7.34 


Thus the random variable V,; has zero mean and variance JT,/2. 

From Equation (7.27), we note that the signal component at the coherent detector 
output (during each bit interval) equals +VE,, where E, is the signal energy per bit. Hence, 
the peak instantaneous power of the signal component is Ez- Accordingly, we may define 
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the output signal-to-noise ratio as the instantaneous peak power E, divided by the variance 
of the equivalent noise component in Equation (7.34). We thus write 


2E; 
JT: 


The average signal power at the receiver input equals E,/T,. We thus define an input signal- 
to-noise ratio as 


(SNR)o = 





(7.35) 


E,/Ts 
J 


Hence, eliminating E,/] between Equations (7.35) and (7.36), we may express the output 
signal-to-noise ratio in terms of the input signal-to-noise ratio as 
2% 


(SNR)o = T (SNR); (7.37) 





(SNR); = (7.36) 





It is customary practice to express signal-to-noise ratios in decibels. To that end, we intro- 
duce a term called the processing gain (PG), which is defined as the gain in SNR obtained 
by the use of spread spectrum. Specifically, we write 


Te 
PG == T 
T, (7.38) 
which represents the gain achieved by processing a spread-spectrum signal over an un- 
spread signal. We may thus write Equation (7.37) in the equivalent form: 


10 logio(SNR)o = 10 logio(SNR); + 3 + 10 logio(PG) dB (7.39) 


The 3-dB term on the right-hand side of Equation (7.39) accounts for the gain in SNR 
that is obtained through the use of coherent detection (which presumes exact knowledge 
of the signal phase by the receiver). This gain in SNR has nothing to do with the use of 
spread spectrum. Rather, it is the last term, 10 logi(PG), that accounts for the processing 
gain. Note that both the processing gain PG and the spread factor N (i.e., PN sequence 
length) equal the ratio T,/T,. Thus, the longer we make the PN sequence (or, correspond- 
ingly, the smaller the chip time T, is), the larger will the processing gain be. 


| 7.6 Probability of Error 


Let the coherent detector output v in the direct-sequence spread BPSK system of Figure 
7.9 denote the sample value of a random variable V. Let the equivalent noise component 
vj produced by external interference denote the sample value of a random variable Vy. 
Then, from Equations (7.23) and (7.27) we deduce that 


V =+VE, + Vy (7.40) 


where E, is the transmitted signal energy per bit. The plus sign refers to sending symbol 
(information bit) 1, and the minus sign refers to sending symbol 0. The decision rule used 
. by the coherent detector of Figure 7.9 is to declare that the received bit in an interval (0, 
T,) is 1 if the detector output exceeds a threshold of zero, and that it is 0 if the detector 
output is less than the threshold; if the detector output is exactly zero, the receiver makes 
a random guess in favor of 1 or 0. With both information bits assumed equally likely, we 


498 


CHAPTER 7 = SPREAD-SPECTRUM MODULATION 


find that (because of the symmetric nature of the problem) the average probability of error 
P, is the same as the conditional probability of (say) the receiver making a decision jn 
favor of symbol 1, given that symbol 0 was sent. That is, 


P, = P(V > 0|symbol 0 was sent) 


= PV, > VE) (7-41) 


Naturally, the probability of error P, depends on the random variable V; defined by Equa- 
tion (7.30). According to this definition, V,; is the sum of N identically distributed random 
variables. Hence, from the central limit theorem, we deduce that for large N, the random 
variable V,, assumes a Gaussian distribution. Indeed, the spread factor or PN sequence 
length N is typically large in the direct-sequence spread-spectrum systems encountered in 
practice, under which condition the application of the central limit theorem is justified, 
Farlier we evaluated the mean and variance of V,;; see Equations (7.32) and (7.34), 
We may therefore state that the equivalent noise component V; contained in the coherent 
detector output may be approximated as a Gaussian random variable with zero mean and 
variance JT,/2, where J is the average interference power and T, is the chip duration. With 
this approximation at hand, we may then proceed to calculate the probability of the event 
V> VE, and thus express the average probability of error in accordance with Equation 


(7.41) as 
P, = 5 ente( že) (7.42) 


This simple formula, which invokes the Gaussian assumption, is appropriate for DS/BPSK 
binary systems with large spread factor N. 


a ANTIJAM CHARACTERISTICS 


It is informative to compare Equation (7.42) with the formula for the average probability 
of error for a coherent binary PSK system reproduced here for convenience of presentation 


[see Equation (6.20)] 
Bee E, : 
P, = 3 ete £) (7.43) 


Based on this comparison, we see that insofar as the calculation of bit error rate in a direct- 
sequence spread binary PSK system is concerned, the interference may be treated as 
wideband noise of power spectral density No/2, defined by 


No IE 


5 A (7.44) 


This relation is simply a restatement of an earlier result given in Equation ( 7.34). 
Since the signal energy per bit E, = PT,, where P is the average signal power and T, 
is the bit duration, we may express the signal energy per bit-to-noise spectral density ratio 


as 
Peg (2 (5 (7.45) 
No \T/\J i 
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Using the definition of Equation (7.38) for the processing gain PG we may reformulate 
this result as 
J_ PG 
P EN, 





(7.46) 


The ratio J/P is termed the jamming margin. Accordingly, the jamming margin and the 
processing gain, both expressed in decibels, are related by 


(Jamming margin)as = (Procesing gain), — 10 losio( =) (7.47) 
: 0/min 
where (E;/No)min is the minimum value needed to support a prescribed average probability 
of error. 


P EXAMPLE 7.3 
A spread-spectrum communication system has the following parameters: 
Information bit duration, T, = 4.095 ms 
PN chip duration, T, = 1 us 
Hence, using Equation (7.38) we find that the processing gain is 
PG = 4095 


Correspondingly, the required period of the PN sequence is N = 4095, and the shift-register 
length is = 12. 

For a satisfactory reception, we may assume that the average probability of error 
is not to exceed 10~°. From the formula for a coherent binary PSK receiver, we find that 
E,/No = 10 yields an average probability of error equal to 0.387 X 107°. Hence, using this 
value for E;/No, and the value calculated for the processing gain, we find from Equation (7.47) 
that the jamming margin is 


(Jamming margin), = 10 logig 4095 — 10 logio(10) 
= 36.1 — 10 
= 26.1 dB 


That is, information bits at the receiver output can be detected reliably even when the noise 
or interference at the receiver input is up to 409.5 times the received signal power. Clearly, 
this is a powerful advantage against interference (jamming), which is realized through the 
clever use of spread-spectrum modulation. k 


| 7.7 Frequency-Hop Spread Spectrum 


In the type of spread-spectrum systems discussed in Section 7.4, the use of a PN sequence 
to modulate a phase-shift-keyed signal achieves instantaneous spreading of the transmis- 
sion bandwidth. The ability of such a system to combat the effects of jammers is determined 
by the processing gain of the system, which is a function of the PN sequence period. The 
processing gain can be made larger by employing a PN sequence with narrow chip dura- 
tion, which, in turn, permits a greater transmission bandwidth and more chips per bit. 
However, the capabilities of physical devices used to generate the PN spread-spectrum 
signals impose a practical limit on thé attainable processing gain. Indeed, it may turn out 
that the processing gain so attained is still not large enough to overcome the effects of 


500 


CHAPTER 7 = SPREAD-SPECTRUM MODULATION 


some jammers of concern, in which case we have to resort to other methods. One such 
alternative method is to force the jammer to cover a wider spectrum by randomly hopping 
the data-modulated carrier from one frequency to the next. In effect, the spectrum of the 
transmitted signal is spread sequentially rather than instantaneously; the term “sequen. 
tially” refers to the pseudo-random-ordered sequence of frequency hops. 

The type of spread spectrum in which the carrier hops randomly from one frequency 
to another is called frequency-hop (FH) spread spectrum. A common modulation format 
for FH systems is that of M-ary frequency-shift keying (MFSK). The combination of these 
two techniques is referred to simply as FH/MFSK. (A description of M-ary FSK is presented 
in Chapter 6.) 

Since frequency hopping does not cover the entire spread spectrum instantaneously, 
we are led to consider the rate at which the hops occur. In this context, we may identify 
two basic (technology-independent) characterizations of frequency hopping: 


1, Slow-frequency hopping, in which the symbol rate R, of the MFSK signal is an integer 
multiple of the hop rate R,. That is, several symbols are transmitted on each fre. 
quency hop. , 

2. Fast-frequency hopping, in which the hop rate R, is an integer multiple of the MFSK 
symbol rate R,. That is, the carrier frequency will change or hop several times during 
the transmission of one symbol. 


Obviously, slow-frequency hopping and fast-frequency hopping are the converse of one 
another. In the foliowing, these two characterizations of frequency hopping are considered 
in turn. 


SLOW-FREQUENCY HOPPING 


Figure 7.102 shows the block diagram of an FH/MFSK transmitter, which involves fre- 
quency modulation followed by mixing. First, the incoming binary data are applied to an 
M-ary FSK modulator. The resulting modulated wave and the output from a digital fre- 
quency synthesizer are then applied to a mixer that consists of a multiplier followed by a 
band-pass filter. The filter is designed to select the sum frequency component resulting 
from the multiplication process as the transmitted signal. In particular, successive k-bit 
segments of a PN sequence drive the frequency synthesizer, which enables the carrier fre- 
quency to hop over 2* distinct values. On a single hop, the bandwidth of the transmitted 
signal is the same as that resulting from the use of a conventional MFSK with an alphabet 
of M = 2* orthogonal signals. However, for a complete range of 2* frequency hops, the 
transmitted FH/MEFSK signal occupies a much larger bandwidth. Indeed, with present-day 
technology, FH bandwidths on the order of several GHz are attainable, which is an order 
of magnitude larger than that achievable with direct-sequence spread spectra. An impli- 
cation of these large FH bandwidths is that coherent detection is possible only within each 
hop, because frequency synthesizers are unable to maintain phase coherence over succes- 
sive hops. Accordingly, most frequency-hop spread-spectrum communication systems use 
noncoherent M-ary modulation schemes. 

In the receiver depicted in Figure 7.10b, the frequency hopping is first removed by 
mixing (down-converting) the received signal with the output of a local frequency synthe- 
sizer that is synchronously controlled in the same manner as that in the transmitter. The 
resulting output is then band-pass filtered, and subsequently processed by a noncoherent 
M-ary FSK detector. To implement this M-ary detector, we may use a bank of M nonco- 
herent matched filters, each of which is matched to one of the MFSK tones. (Noncoherent 
matched filters are described in Chapter 6.) An estimate of the original symbol transmitted 
is obtained by selecting the largest filter output. 
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FIGURE 7.10 Frequency-hop spread M-ary frequency-shift keying. (a) Transmitter. (b) Receiver. 


An individual FH/MFSK tone of shortest duration is referred to as a chip; this ter- 
minology should not be confused with that used in Section 7.4 describing DS/BPSK. The 
chip rate, R., for an FH/MFSK system is defined by 


R. = max(R,, R,) (7.48) 


where R, is the bop rate, and R, is the symbol rate. 

A slow FH/MFSK signal is characterized by having multiple symbols transmitted per 
hop. Hence, each symbol of a slow FH/MFSK signal is a chip. Correspondingly, in a slow 
FH/MFSK system, the bit rate R of the incoming binary data, the symbol rate R, of the 
MEFSK signal, the chip rate R,, and the hop rate R, are related by 


R,=R, = =R, (7.49) 


where K = log, M. 

At each hop, the MFSK tones are separated in frequency by an integer multiple of 
the chip rate R, = R., ensuring their orthogonality. The implication of this condition is 
that any transmitted symbol will not produce any crosstalk in the other M — 1 noncoherent 
matched filters constituting the MFSK detector of the receiver in Figure 7.10b. By “cross- 
talk” we mean the spillover from one filter output into an adjacent one. The resulting 
performance of the slow FH/MFSK system is the same as that for the noncoherent detection 
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of conventional (unhopped) MFSK signals in additive white Gaussian noise. Thus the 
interfering (jamming) signal has an effect on the FH/MFSK receiver, in terms of average 
probability of symbol error, equivalent to that of additive white Gaussian noise on 4 
conventional noncoherent M-ary FSK receiver experiencing no interference. On the basig 
of this equivalence, we may use Equation (6.140) for approximate evaluation of the prob. 
ability of symbol error in the FH/MFSK system. 

Assuming that the jammer decides to spread its average power J over the entire 
frequency-hopped spectrum, the jammer’s effect is equivalent to an AWGN with power 
spectral density No/2, where No = J/W,and W, is the FH bandwidth. The spread-spectrum 
system is thus characterized by the symbol energy-to-noise spectral density ratio: 


E Pi 
No WAR, 
where the ratio P/J is the reciprocal of the jamming margin. The other ratio in the denom- 


inator of Equation (7.50) is the processing gain of the slow FH/MFSK system, which is 
defined by 





(7.50) 


_ ©. 


R, 
2 (7.51) 


PG 


That is, the processing gain (expressed in decibels) is equal to 10 logy 2* = 3k, where k 
is the length of the PN segment employed to select a frequency hop. 

This result assumes that the jammer spreads its power over the entire FH spectrum. 
However, if the jammer decides to concentrate on just a few of the hopped frequencies, 
then the processing gain realized by the receiver would be less than 3k decibels. 


b> EXAMPLE 7.4 


Figure 7.11q illustrates the variation of the frequency of a slow FH/MFSK signal with time 
for one complete period of the PN sequence. The period of the PN sequence is 2* ~ 1 = 15. 
The FH/MFSK signal has the following parameters: 


Number of bits per MFSK symbol K=2 


Number of MFSK tones M=2K =4 
Length of PN segment per hop k=3 
Total number of frequency hops 2 = 


In this example, the carrier is hopped to a new frequency after transmitting two symbols or 
equivalently, four information bits. Figure 7.114 also includes the input binary data, and the 
PN sequence. controlling the selection of FH carrier frequency. It is noteworthy that although 
there are eight distinct frequencies available for hopping, only three of them are utilized by 
the PN sequence. 

Figure 7.11b shows the variation of the dehopped frequency with time. This variation 
is recognized to be the same as that of a conventional MFSK signal produced by the given 
input data. 4 


FAST-FREQUENCY HOPPING 


A fast FH/MFSK system differs from a slow FH/MFSK system in that there are multiple 
hops per M-ary symbol. Hence, in a fast FH/MFSK system, each hop is a chip. In general, 
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FIGURE 7.11 Illustrating slow-frequency hopping. (a) Frequency variation for one complete 
period of the PN sequence. (b) Variation of the dehopped frequency with time. 


fast-frequency hopping is used to defeat a smart jammer’s tactic that involves two func- 
tions: measurements of the spectral contént of the transmitted signal, and retuning of the 
interfering signal to that portion of the frequency band. Clearly, to overcome the jammer, 
the transmitted signal must be hopped to a new carrier frequency before the jammer is 
able to complete the processing of these two functions. 

For data recovery at the receiver, noncoherent detection is used. However, the de- 
tection procedure is quite different from that used in a slow FH/MFSK receiver. In partic- 
ular, two procedures may be considered: 


1. For each FH/MFSK symbol, separate decisions are made on the K frequency-hop 


chips received, and a simple rule based on majority vote is used to make an estimate 
of the dehopped MFSK symbol. 


2. For each FH/MFSK symbol, likelihood functions are computed as functions of the 
total signal received over K chips, and the largest one is selected. 
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A receiver based on the second procedure is optimum in the sense that it minimizes the 


average probability of symbol error for a given E,/No. 


$ EXAMPLE 7.5 


Figure 7.124 illustrates the variation of the transmitted frequency of a fast FH/MFSK signal 


with time. The signal has the following parameters: 


Number of bits per MFSK symbol 


Number of MFSK tones 
Length of PN segment per hop 
Total number of frequency hops 


K=2 
M=2=4 
k=3 
2k =8 


In this example, each MFSK symbol has the same number of bits and chips; that is, the chip 


rate R, is the same as the bit rate Rp. After each chip, 


the carrier frequency of the transmitted 


MESK signal is hopped to a different value, except for few occasions when the k-chip segment 


of the PN sequence repeats itself. 
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FIGURE 7.12 
time. (b) Variation of the dehopped frequency with time. 


Illustrating fast-frequency hopping. (a) Variation of the transmitter frequenc 
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Figure 7.126 depicts the time variation of the frequency of the dehopped MFSK signal, 
which is the same as that in Example 7.4. < 


7.8 Computer Experiments: 
Maximal-Length and Gold Codes 


Code-division multiplexing (CDM) provides an alternative to the traditional methods of 
frequency-division multiplexing (FDM) and time-division multiplexing (TDM). It does not 
require the bandwidth allocation of FDM (discussed in Chapter 2) nor the time synchro- 
nization needed in TDM (discussed in Chapter 3). Rather, users of a common channel are 
permitted access to the channel through the assignment of a “spreading code” to each 
individual user under the umbrella of spread-spectrum modulation. The purpose of this 
computer experiment is to study a certain class of spreading codes for CDM systems that 
provide a satisfactory performance. 

In an ideal CDM system, the cross-correlation between any two users of the system 
is zero. For this ideal condition to be realized, we require that the cross-correlation function 
between the spreading codes assigned to any two users of the system be zero for all cyclic 
shifts. Unfortunately, ordinary PN sequences do not satisfy this requirement because of 
their relatively poor cross-correlation properties. 

As a remedy for this shortcoming of ordinary PN sequences, we may use a special 
class of PN sequences called Gold sequences (codes),* the generation of which is embodied 
in the following theorem: 


Let g,(X) and g>(X) be a preferred pair of primitive polynomials of degree n whose 
corresponding shift registers generate maximal-length sequences of period 2” — 1 
and whose cross-correlation function has a magnitude less than or equal to 


2erne N i for n odd (7.52) 


Or 


girt2v2 4 4 for n even and n # 0 mod 4 (7.53) 


Then the shift register corresponding to the product polynomial g,(X) - g2(X) will 
generate 2” + 1 different sequences, with each sequence having a period of 2” — 1, 
and the cross-correlation between any pair of such sequences satisfying the preced- 
ing condition. 


Hereafter, this theorem is referred to as Gold’s theorem. 

To understand Gold’s theorem, we need to define what we mean by a primitive 
polynomial. Consider a polynomial g(X) defined over a binary field (i.e., a finite set of two 
elements, 0 and 1, which is governed by the rules of binary arithmetic). The polynomial 
g(X) is said to be an irreducible polynomial if it cannot be factored using any polynomials 
from the binary field. An irreducible polynomial g(X) of degree m is said to be a primitive 
polynomial if the smallest integer m for which the polynomial g(X) divides the factor 
X” + Lis# = 2” —1. Further discussion of this topic is deferred to Chapter 8; in particular, 
see Example 8.3. 
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Experiment 1. Correlation Properties of PN Sequences 


Consider a pair of shift registers for generating two PN sequences of period 27 — 1 = 127 
One feedback shift register has the feedback taps [7, 1] and the other one has the feedback 
taps [7, 6, 5, 4]. Both sequences have the same autocorrelation function shown in Figure 
7.13a, which follows readily from the definition presented in Equation (7.5). 

However, the calculation of the cross-correlation function between PN sequences jg 
a more difficult proposition, particularly for large n. To perform this calculation, we reson 
to the use of computer simulation for varying cyclic shift 7 inside the interval 
0 < 7 = 2" — 1. The results of this computation are presented in Figure 7.130. This figure 
confirms the poor cross-correlation property of PN sequences compared to their autocor- 
relation function. The magnitude of the cross-correlation function exceeds 40. 
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FIGURE 7.13 (a) Autocorrelation function R,(7), and (b) cross-correlation function R,2(7) of the 
two PN sequences [7, 1] and [7, 6, 5, 4]. 
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FIGURE 7.14 Generator for a Gold sequence of period 2” — 1 = 127. 


Experiment 2. Correlation Properties of Gold Sequences 


For our next experiment, we consider Gold sequences with period 27 — 1 = 127. To 
generate such a sequence for n = 7 we need a preferred pair of PN sequences that satisfy 
Equation (7.52) (n odd), as shown by 


2eene 44 =244+1=17 
This requirement is satisfied by the PN sequences with feedback taps [7, 4] and [7, 6, 5, 4]. 
The Gold-sequence generator is shown in Figure 7.14 that involves the modulo-2 addition 
of these two sequences. According to Gold’s theorem, there are a total of 
2"+1=27+1= 129 
sequences that satisfy Equation (7.52). The cross-correlation between any pair of such 


sequences is shown in Figure 7.15, which is indeed in full accord with Gold’s theorem. In 
particular, the magnitude of the cross-correlation is less than or equal to 17. 





20 T T T — 


ISTH y II 


m 
O 


ot 
T 





7 
= 

















Cross-correlation function Ry o(7) 
o 


SS 
——— 
L 
i 1 L 





























































































































oa 
5 

| i i 
-10 

| | | | | 
i | 1 il! i 
-20 ‘ , es, 

-100 -50 0 50 100 


Delay 7 


FIGURE 7.15 Cross-correlation function R;2(7) of a pair of Gold sequences based on the two 
PN sequences [7, 4] and [7, 6, 5, 4]. 
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1 7.9 Summary and Discussion 


Direct-sequence M-ary phase shift keying (DS/MPSK) and frequency-hop M-ary frequency 
shift-keying (FH/MFSK) represent two principal categories of spread-spectrum commy. 
nications. Both of them rely on the use of a pseudo-noise (PN) sequence, which is applied 
differently in the two categories. 

In a DS/MPSK system, the PN sequence makes the transmitted signal assume a 
noiselike appearance by spreading its spectrum over a broad range of frequencies simul- 
taneously. For the phase-shift keying, we may use binary PSK (i.e., M = 2) with a single 
carrier. Alternatively, we may use QPSK (i.e., M = 4), in which case the data are trans. 
mitted using a pair of carriers in phase quadrature. (Both PSK and QPSK are discussed in 
Section 6.3.) The usual motivation for using QPSK is to provide for improved bandwidth 
efficiency. In a spread-spectrum system, bandwidth efficiency is usually not of prime con- 
cern. Rather, the use of QPSK is motivated by the fact that it is less sensitive to some types 
of interference (jamming). 

In an FH/MESK system, the PN sequence makes the carrier hop over a number of 
frequencies in a pseudo-random manner, with the result that the spectrum of the trans- 
mitted signal is spread in a sequential manner. 

Naturally, the direct-sequence and frequency-hop spectrum-spreading techniques 
may be employed in a single system. The resulting system is referred to as hybrid DS/FH 
spread-spectrum system. The reason for seeking a hybrid approach is that advantages of 
both the direct-sequence and frequency-hop spectrum-spreading techniques are realized in 
the same system. 

A discussion of spread-spectrum communications would be incomplete without some 
reference to jammer waveforms. The jammers encountered in practice include the follow- 


ing types: 


1. The barrage noise jammer, which consists of band-limited white Gaussian noise of 
high average power. The barrage noise jammer is a brute-force jammer that does not 
exploit any knowledge of the antijam communication system except for its spread 
bandwidth. 

2. The partial-band noise jammer, which consists of noise whose total power is evenly 
spread over some frequency band that is a subset of the total spread bandwidth. 
Owing to the smaller bandwidth, the partial-band noise jammer is easier to generate 
than the barrage noise jammer. 

3. The pulsed noise jammer, which involves transmitting wideband noise of power 


p 


Jpeak = 


for a fraction p of the time, and nothing for the remaining fraction 1 — p of the time. 
The average noise power equals J. 

4, The single-tone jammer, which consists of a sinusoidal wave whose frequency lies 
inside the spread bandwidth; as such, it is the easiest of all jamming signals to 
generate. 

5. The multitone jammer, which is the tone equivalent of the partial-band noise jammer. 


In addition to these five, many other kinds of jamming waveforms occur in practice. Jn” 
any event, there is no single jamming waveform that is worst for all spread-spectru® 
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systems, and there is no single spread-spectrum system that is best against all possible 
jamming waveforms. 


i NOTES AND REFERENCES 


1. The definition of spread-spectrum modulation presented in the Introduction is adapted from 
Pickholtz, Schilling, and Milstein (1982). This paper presents a tutorial review of the theory 
of spread-spectrum communications. 

For introductory papers on the subject, see Viterbi (1979), and Cook and Marsh 
(1983). For books on the subject, see Dixon (1984), Holmes (1982), Ziemer and Peterson 
(1985, pp. 327-649), Cooper and McGillem (1986, pp. 269-411), and Simon, Omura, 
Scholtz, and Levitt (1985, Volumes I, II, and M). The three-volume book by Simon et al. is 
the most exhaustive treatment of spread-spectrum communications available in the open 
literature. The development of spread-spectrum communications dates back to about the 
mid-1950s. For a historical account of these techniques, see Scholtz (1982). This latter paper 
traces the origins of spread-spectrum communications back to the 1920s. Much of the his- 
torical materia! presented in this paper is reproduced in Chapter 2, Volume I, of the book 
by Simon et al. 

The book edited by Tantaratana and Ahmed (1998) includes introductory and ad- 
vanced papers on wireless applications of spread-spectrum modulation. The papers are 
grouped into the following categories: spread-spectrum technology, cellular mobile systems, 
satellite communications, wireless local area networks, and global positioning systems 
(GPS). 


2. For further details on maximal-length sequences, see Golomb (1964, pp. 1-32), Simon, 
Omura, Scholtz, and Levitt (1985, pp. 283-295), and Peterson and Weldon (1972). The 
last reference includes an extensive list of polynomials for generating maximal-length se- 
quences; see also Dixon (1984). For a tutorial paper on pseudo-noise sequences, see Sarwate 
and Pursley (1980). 


3. Table 7.1 is extracted from the book by Dixon (1984, pp. 81-83), where feedback connec- 
tions of maximal-length sequences are tabulated for shift-register length m extending up 
to 89. 


4. For detailed discussion of the synchronization problem in spread-spectruin communications, 
see Ziemer and Peterson (1985, Chapters 9 and 10) and Simon et al. (1985, Volume III). 


5. The original papers on Gold sequences are Gold (1967, 1968). A detailed discussion of Gold 
sequences is presented in Holmes (1982). 


i PROBLEMS 


Pseudo-Noise Sequences 
7.1 A pseudo-noise (PN) sequence is generated using a feedback shift register of length 
m = 4, The chip rate is 10’ chips per second. Find the following parameters: 
(a) PN sequence length. 
(b) Chip duration of the PN sequence. 
(c) PN sequence period. 
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7.2 Figure P7.2 shows a four-stage feedback shift register. The initial state of the register jg 
1000. Find the output sequence of the shift register. 
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FIGURE P7.2 


7.3 For the feedback shift register given in Problem 7.2, demonstrate the balance property 
and run property of a PN sequence. Also, calculate and plot the autocorrelation function 
of the PN sequence produced by this shift register. 

7.4 Referring to Table 7.1, develop the maximal-length codes for the three feedback config- 
urations [6, 1], [6, 5, 2, 1], and [6, 5, 3, 2], whose period is N = 63. 

7.5 Figure P7.5 shows the modular multitap version of the linear feedback shift-register 
shown in Figure 7.4b. Demonstrate that the PN sequence generated by this scheme is 
exactly the same as that described in Table 7.2b. 
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Clock 


FIGURE P7.5 


Direct Sequence/Phase-Shift Keying System 


7.6 Show that the truth table given in Table 7.3 can be constructed by combining the follow- 
ing two steps: 
(a) The message signal b(t) and PN signal c(t) are added modulo-2. 
(b) Symbols 0 and 1 at the modulo-2 adder output are represented by phase shifts of 0 
and 180 degrees, respectively. 
7.7 A single-tone jammer 
jt) = V2] cos(2nf.t + 9) 
is applied to a DS/BPSK system. The N-dimensional transmitted signal x(t) is described 
by Equation (7.16). Find the 2N coordinates of the jammer /(#). 


7.8 The processing gain of a spread-spectrum system may be expressed as the ratio of the 
spread bandwidth of the transmitted signal to the despread bandwidth of the received 
signal. Justify this statement for the DS/BPSK system. 
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7.9 A direct-sequence spread binary phase-shift keying system uses a feedback shift register 
of length 19 for the generation of the PN sequence. Calculate the processing gain of the 
system. 

7.10 In a DS/BPSK system, the feedback shift register used to generate the PN sequence has 
length m = 19. The system is required to have an average probability of symbol error 
due to externally generated interfering signals that does not exceed 1075. Calculate the 

_ following system parameters in decibels: 
(a) Processing gain. 
(b) Antijam margin. 

7.11 In Section 7.5, we presented an analysis on the signal-space dimensionality and processing 
gain of a direct sequence spread-spectrum system using binary phase-shift keying. Extend 
the analysis presented therein to the case of such a system using quadriphase-shift keying. 


Frequency-Hop Spread Spectrum 


7.12 A slow FH/MEFSK system has the following parameters: 
Number of bits per MFSK symbol = 4 
Number of MFSK symbols per hop = 5 
Calculate the processing gain of the system. 
7.13 A fast FH/MFSK system has the following parameters: 
Number of bits per MFSK symbol = 4 
Number of hops per MFSK symbol = 4 
Calculate the processing gain of the system. 


Computer Experiments 


7.14 Consider two PN sequences of period N = 63. One sequence has the feedback taps [6, 1] 
and the other sequence has the feedback taps [6, 5, 2, 1], which are picked in accordance 
with Table 7.1. 
(a) Compute the autocorrelation function of these two sequences, and their cross- 
correlation function. 
(b) Compare the cross-correlation function computed in part (a) with the cross- 
correlation function between the sequence [6, 5 2, 1] and its mirror image [6, 5, 4, 1]. 
Comment on your results. 
7.15 (a) Compute the partial cross-correlation function of a PN sequence with feedback taps 
[5, 2] and its image sequence defined by the feedback taps [5, 3]. 
(b) Repeat the computation for the PN sequence with feedback taps [5, 2] and the PN 
sequence with feedback taps [5, 4, 2, 1]. 
{c) Repeat the computation for the PN sequence with feedback taps [5, 4, 3, 2] and the 
PN sequence with feedback taps [5, 4, 2, 1]. 
The feedback taps [5, 2], [5, 4, 3, 2], and [5, 4, 2, 1] are possible taps for a maximal- 
length sequence of period 31, in accordance with Table 7.1. 





MULTIUSER RADIO 
COMMUNICATIONS 


As its name implies, multiuser communications refers to the simultaneous use of a 
communication channel by a number of users. In this chapter, we discuss multiuser 
communication systems that rely on radio propagation for linking the receivers to the 
transmitters. 

In particular, we focus on the following topics: 


> Multiple-access techniques, which are basic to multiuser communication systems. 
> Satellite communications, offering global coverage. 


> Radio link analysis, highlighting the roles of transmitting and receiving antennas and free- 
space propagation. l 


> Wireless communications with emphasis on mobility and the multipath phenomenon. 
> Speech coding for wireless communications. 


> Adaptive antennas for wireless communications. 


i 8.1 Introduction 


Much of the material on communication theory presented in earlier chapters has been 
based ona particular idealization of the communication channel, namely, a channel model 
limited in bandwidth and corrupted by additive white Gaussian noise (AWGN). The clas- 
sical communication theory so developed is mathematically elegant, providing a sound 
introduction to the ever-expanding field of communication systems. An example of a phys- 
ical channel that is well represented by such a model is the satellite communications chan- 
nel. It is therefore befitting that the first type of multiuser communications discussed in 
this chapter is satellite communications. 

A satellite communication system in geostationary orbit relies on line-of-sight radio 
propagation for the operation of its uplink from an earth terminal to the transponder and 
the downlink from the transponder to another earth terminal. Thus the discussion of 
satellite communications naturally leads to the analysis of radio propagation in free space, 
linking a receiving antenna to a transmitting antenna. 

The use of satellite communications offers global coverage. The other multiuser com 
munication system studied in this chapter, namely, wireless communications, offers mo- 
bility which, in conjunction with existing telephone networks and satellite communication 
systems, permits a mobile unit to communicate with anyone, anywhere in the world. An- 
other characteristic feature of wireless communication systems is that they are tetherless 
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(i.e., total freedom of location is permitted), hence the interest in their use for local area 
networks (i.e., data networks confined to buildings up to a few kilometers in size) due to 
significant advantages over conventional cabling: elimination of wiring and rewiring, flex- 
ibility of creating new communication services, and mobility of users. 

The radio propagation channel characterizing wireless communications deviates 

` from the idealized AWGN channel model due to the presence of multipath, which is a 
non-Gaussian form of signal-dependent phenomenon that arises because of reflections of 
the transmitted signal from fixed and moving objects. The presence of multipath raises 
practical difficulties in the use of a radio propagation channel and complicates its math- 
ematical analysis. Simply put, multipath is a physical phenomenon that is intrinsic to the 
operation of indoor and outdoor forms of wireless communications. 

Before proceeding to discuss specific aspects of satellite communications and wireless 
communications, however, it is appropriate that we begin the discussion by describing 
multiple-access techniques, which enable different users to simultaneously (or nearly so) 
access a common channel. 


i 8.2 Multiple-Access Techniques 


Multiple access is a technique whereby many subscribers or local stations can share the 
use of a communication channel at the same time or nearly so, despite the fact that their 
individual transmissions may originate from widely different locations. Stated in another 
way, a multiple-access technique permits the communication resources of the channel to 
be shared by a large number of users seeking to communicate with each other. 

There are subtle differences between multiple access and multiplexing that should be 
noted: 


» Multiple access refers to the remote sharing of a communication channel such as a 
satellite or radio channel by users in highly dispersed locations. On the other hand, 
multiplexing refers to the sharing of a channel such as a telephone channel by users 
confined to a local site. 

> In a multiplexed system, user requirements are ordinarily fixed. In contrast, in a 
multiple-access system user requirements can change dynamically with time, in which 
case provisions are necessary for dynamic channel allocation. 


For obvious reasons it is desirable that in a multiple-access system the sharing of 
resources of the channel be accomplished without causing serious interference between 
users of the system. In this context, we may identify four basic types of multiple access: 


1. Frequency-division multiple access (FDMA). 

In this technique, disjoint subbands of frequencies are allocated to the different users on 
a continuous-time basis. In order to reduce interference between users allocated adjacent 
channel bands, guard bands are used to act as buffer zones, as illustrated in Figure 8.14. 
These guard bands are necessary because of the impossibility of achieving ideal filtering 
for separating the different users. 


2. Time-division multiple access (TDMA). 

In this second technique, each user is allocated the full spectral occupancy of the channel, 
but only for a short duration of time called a time slot. As shown in Figure 8.16, buffer 
zones in the form of guard times are inserted between the assigned time slots. This is done 
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FicurE 8.1 Illustrating the ideas behind multiple-access techniques. (a) Frequency-division 
multiple access. (b) Time-division multiple access. (c) Frequency-hop multiple access. 


to reduce interference between users by allowing for time uncertainty that arises due to 
system imperfections, especially in synchronization schemes. 


3. Code-division multiple access (CDMA). 

In FDMA, the resources of the channel are shared by dividing them along the frequency 
coordinate into disjoint frequency bands, as illustrated in Figure 8.1a. In TDMA, the 
resources are shared by dividing them along the time coordinate into disjoint time slots, 
as illustrated in Figure 8.1b. In Figure 8.1c, we illustrate another technique for sharing the 
channel resources by using a hybrid combination of FOMA and TDMA, which represents 
a specific form of code-division multiple access (CDMA). For example, frequency hopping 
may be employed to ensure that during each successive time slot, the frequency bands 
assigned to the users are reordered in an essentially random manner. To be specific, during 
time slot 1, user 1 occupies frequency band 1, user 2 occupies frequency band 2, user 3 
occupies frequency band 3, and so on. During time slot 2, user 1 hops to frequency band 
3, user 2 hops to frequency band 1, user 3 hops to frequency band 2, and so on. Such an 
arrangement has the appearance of the users playing a game of musical chairs. An impor- 
tant advantage of CDMA over both FOMA and TDMA is that it can provide for secure 
communications. In the type of CDMA illustrated in Figure 8.1c, the frequency hopping 
mechanism can be implemented through the use of a pseudo-noise (PN) sequence. 


4, Space-division multiple access (SDMA). 

In this multiple-access technique, resource allocation is achieved by exploiting the spatial 
separation of the individual users. In particular, multibeam antennas are used to separate 
radio signals by pointing them along different directions. Thus, different users are enabled 
to access the channel simultaneously on the same frequency or in the same time slot. 


These multiple-access techniques share a common feature: allocating the communication 
resources of the channel through the use of disjointedness (or orthogonality in a loose 
sense) in time, frequency, or space. 

With this background material at hand, we are now ready to discuss some important 
multiuser communication systems. 


| 8.3 Satellite Communications 























In a geostationary satellite communication system,’ a message signal is transmitted from 
an earth station via an uplink to a satellite, amplified in a transponder (i.e., electron’ 
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FIGURE 8.2 Satellite communications system. 


circuitry) on board the satellite, and then retransmitted from the satellite via a downlink 
to another earth station, as illustrated in Figure 8.2. The most popular frequency band for 
satellite communications is 6 GHz (C-band) for the uplink and 4 GHz for the downlink. 
The ise of this frequency band offers the following advantages: 


> Relatively inexpensive microwave equipment. 


> Low attenuation due to rainfall; rainfall is the primary atmospheric cause of signal 
degradation. 
» Insignificant sky background noise; the sky background noise (due to random noise 


emissions from galactic, solar, and terrestrial sources) reaches its lowest level between 
1 and 10 GHz. 


However, radio interference limits the applications of communication satellites operating 
in the 6/4 GHz band, because the transmission frequencies of this band coincide with those 
used for terrestrial microwave systems. This problem is eliminated in the more powerful 
“second-generation” communication satellites that operate in the 14/12 GHz band (i.e. 
Ku-band); moreover, the use of these higher frequencies makes it possible to build smaller 
and therefore less expensive antennas. 

The block diagram of Figure 8.3 shows the basic components of a single transponder 
channel of a typical communication satellite. Specifically, the receiving antenna output of 
the uplink is applied to the cascade connection of the following components: 


» Band-pass filter, designed to separate the received signal from among the different 
radio channels. 


> Low-noise amplifier. 
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FIGURE 8.3 Block diagram of transponder. 
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> Frequency down-converter, the purpose of which is to convert the received radio 
frequency (RF) signal to the desired downlink frequency. 

> Traveling-wave tube amplifier, which provides high gain over a wide band of fre. 
quencies. In a traveling-wave tube (TWT), an electromagnetic signal travels along a 
helix (i.e., a spring-shaped coil of wire), while electrons in a high-voltage beam travel 
through the helix at a speed close to that of the signal wave; the net result is the 
transfer of power from the electrons to the wave, which grows rapidly as the signal 
wave travels down the helix. 


The channel configuration shown in Figure 8.3 uses a single frequency translation. Other 
channel configurations do the frequency conversion from the uplink to the downlink fre. 
quency in two stages: down-conversion to an intermediate frequency, followed by ampli- 
fication, and then up-conversion to the desired transmit frequency. 

Propagation time delay becomes particularly pronounced in a satellite channel be- 
cause of the large distances involved. Specifically, speech signals sent by satellite incur a 
transmission delay of approximately 270 ms. Hence, for speech signals, any impedance 
mismatch at the receiving end of a satellite link results in an echo of the speaker’s voice, 
which is heard back at the transmitting end after a round-trip delay of approximately 540 
ms. We may overcome this problem by using an echo canceller, which is a device that 
subtracts an estimate of the echo from the return path; elimination of the echo is performed 
by means of a special filter that adapts itself to the changing channel characteristics. 

The satellite channel is closely represented by an additive white Gaussian noise 
(AWGN) model, which applies to both the uplink and downlink portions of the satellite 
communication system. Accordingly, much of the material presented in Chapter 6 on 
passband systems for the transmission of data, with particular reference to phase-shift 
keying and frequency-shift keying techniques, is directly applicable to digital satellite 
communications. 

A satellite transponder differs from a conventional microwave line-of-sight repeater 
in that many earth stations can access the satellite from widely different locations on earth 
at the same time or nearly so. This capability is made possible by using one of the multiple- 
access techniques discussed in Section 8.2. In this context we may offer the following 
observations: 


» Ina satellite channel, nonlinearity of the transponder is the primary cause of inter- 
ference between users. To contain this serious problem, the traveling-wave tube am- 
plifier in the transponder is purposely operated below capacity. Consequently, we 
find that in an FDMA system the power efficiency of the system is reduced because 
of the necessary power backoff of the traveling-wave tube amplifier. 

> Ina TDMA system, the users access the satellite transponder one at a time. Accord- 
ingly, the satellite transponder is now able to operate close to full power efficiency 
by permitting the traveling-wave tube amplifier to run into saturation. This, in tum, 
means that TDMA uses the transponder more efficiently than FOMA, hence its wide 
use in the implementation of digital satellite communication systems. 

> SDMA operates by exploiting the spatial locations of earth stations, which is 
achieved by means of onboard switching. Specifically, the transponder is equipped 
with multiple antennas, with the proper antenna beam being selected for radio trans- 
mission to the particular earth station demanding use of the transponder. 


In addition to multiple access, another capability of a satellite channel is that of 
broadcasting with emphasis on broad area coverage. Here we mention broadcasting sat 
ellites, which are characterized by their high power transmission to inexpensive receivers. 
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This characteristic is exploited in the use of direct broadcast satellites (DBS), designed for 
home reception of television services on a very wide scale. By comparison with the large 
earth stations used for satellite communications, the earth stations for DBS are very simple 
and therefore inexpensive. 


j 8.4 Radio Link Analysis 


An important issue that arises in the design of satellite communication systems is that of 
link budget analysis.? As its name implies, a link budget, or more precisely “link power 
budget,” is the totaling of all the gains and losses incurred in operating a communication 
link. In particular, the balance sheet constituting the link budget provides a detailed ac- 
counting of three broadly defined items: 


1. Apportionment of the resources available to the transmitter and the receiver. 
2. Sources responsible for the loss of signal power. 
3. Sources of noise. 


Putting all these items together into the link budget, we end up with an estimation pro- 
cedure for evaluating the performance of a radio link, which could be the uplink or down- 
link of a satellite communication system. Needless to say, the essence of the communication 
link analysis presented in this section also applies to other radio links that rely on line of 
sight for their operation. It is for this reason the treatment of radio link analysis presented 
in this section is of a generic nature. The section finishes with an illustrative example on 
the budget analysis of a downlink of a digital satellite communication system. 

From the material presented in Chapter 6 we learned that the performance of a digital 
communication system, in the presence of channel noise modeled as additive white Gaus- 
sian noise, is defined by a formula having the shape of a “waterfall” curve as shown in 
Figure 8.4. This figure portrays the probability of symbol error, P,, plotted versus the bit 
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Figure 8.4 “Waterfall” curve relating the probability of error to the E;,/No ratio. 
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energy-to-noise spectral density ratio, E,/No. Once a modulation scheme has been chosen, 
the first design task is to specify two particular values of E,/No as described here; ; 


1. Required E,/No. 

Suppose for example, the prescribed probability of symbol error is P, = 107°. Using the 
waterfall curve of Figure 8.4 pertaining to the modulation scheme of interest, the B/N, 
required to realize the prescribed P, is determined. Let (E,/No)req denote the value of 
E,/No obtained from this calculation. The prescribed P, and the calculated (Ez/No) req de- 
fine a point on the waterfall curve of Figure 8.4, which is designated as operating point 1, 


2. Received E,/No. 

To assure reliable operation of the communication link, the link budget includes a safety 
measure called the dink margin. The link margin provides protection against change and 
the unexpected. Thus the (E,/No) actually received by the system is somewhat larger than 
(Ex/No)req: Let (Ep/No)rcc denote the actual or received E,/No, which defines a second point 
on the waterfall curve of Figure 8.4, designated as operating point 2. The P, corresponding 
to operating point 2 is shown as 107° in Figure 8.4 merely for the purpose of illustration, 
In any event, introducing the link margin denoted by M, we may write 


Fo) -y| Fe 
(i). (i). ey 


Equivalently, expressing the two E,/No values of interest in decibels, we may define the 


link margin as 
E, E, 
MidB) = (2) (aB) - ( 2 
(dB) (E) e ) (Fe) cy (8.2) 


Clearly, the larger we make the link margin M, the more reliable is the communication 
link. However, the increased reliability of the link is attained at the cost of a higher 
E,/No. 


E FREE-SPACE PROPAGATION MODEL 


The next step in formulating the link budget is to calculate the received signal power. 
Naturally, this calculation accounts for all the gains and losses incurred in the transmission 
and reception of the carrier. 

In a radio communication system, the propagation of the modulated signal is accom- 
plished by means of a transmitting antenna, the function of which is twofold: 


> To convert the electrical modulated signal into an electromagnetic field. In this ca- 
pacity, the transmitting antenna acts as an “impedance-transforming” transducer, 
matching the impedance of the antenna to that of free space. 

» To radiate the electromagnetic energy in desired directions. 


At the receiver, we have a receiving antenna whose function is the opposite of that of the 
transmitting antenna: It converts the electromagnetic field into an electrical signal from 
which the modulated signal is extracted. In addition, the receiving antenna may be required 
to suppress radiation originating from directions where it is not wanted. 

Typically, the receiver is located in the farfield of the transmitting antenna, in which 
case, for all practical purposes, we may view the transmitting antenna as a fictitious vol- 
umeless emitter or point source. A complete description of the far field of the point source 
requires knowledge of the electromagnetic field as a function of both time and space: 
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However, insofar as link calculations are concerned, such a complete knowledge is not 
necessary. Rather, it is sufficient to merely specify the variation of the power density for 
the antenna. 

By definition, the Poynting vector or power density is the rate of energy flow per unit 
area; it has the dimensions of watts per square meter. The treatment of the transmitting 
antenna as a point source greatly simplifies matters in that the power density of a point 
source has only a radial component; that is, the radiated energy streams from the source 
along radial lines. l 

It is useful to have a “reference” antenna against which the performance of the 
transmitting and receiving antennas can be compared. The customary practice is to assume 
that the reference antenna is an isotropic source, defined as an omnidirectional (i.e., com- 
pletely nondirectional) antenna that radiates uniformly in all directions. An isotropic 
source is hypothetical because, in reality, all radio antennas have some directivity, however 
small. Nonetheless, the notion of an isotropic source is useful, especially for gain compar- 
ison purposes. 5 

Consider then an isotropic source radiating a total power denoted by P,, measured 
in watts. The radiated power passes uniformly through a sphere of surface area 4rd’, 
where d is the distance (in meters) from the source. Hence, the power density, denoted by 
p(d), at any point on the surface of the sphere is given by 


watts/m? (8.3) 





P, 
pid) = d 


And? 
Equation (8.3) states that the power density varies inversely as the square of the distance 
from a point source. This statement is the familiar inverse-square law that governs the 
propagation of electromagnetic waves in free space. 

Multiplying the power density p(d) by the square of the distance d at which it is 
measured, we get a quantity called radiation intensity denoted by ©. We may thus write 


® = d’p(d) (8.4) 


Whereas the power density p(d) is measured in watts per square meter, the radiation 
intensity ® is measured in watts per unit solid angle (watts per steradian). 

In the case of a typical transmitting or receiving radio antenna, the radiation intensity 
is a function of the spherical coordinates @ and ¢ defined in Figure 8.5. Thus, in general, 
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FIGURE 8.5 Illustrating the spherical coordinates of a point source. 
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we may express the radiation intensity as D(6, $), and so speak of a radiation-intensip 
pattern. The power radiated inside an infinitesimal solid angle dQ. is given by (4, $) do, 
where (referring to Figure 8.5) i 


dQ = sin@ d8 dọ steradians (8.5) 


The total power radiated is therefore 
P= Í (0, p) dQ watts (8.6) 


which is a mathematical statement of the power theorem. In words, the power theorem 
states that if the radiation-intensity pattern (0, $) is known for all values of angle pair 
(6, $), then the total power radiated is given by the integral of ®(0, $) over a solid angle 
of 47 steradians. The average power radiated per unit solid angle is 


1 
Pu =z | oi, 4) da Re 


watts/steradian 


4r 


which represents the radiation intensity that is produced by an isotropic source radiating 
the same total power P. 


Directive Gain, Directivity, and Power Gain’ 


Now the ability of an antenna to concentrate the radiated power in a given direction 
as in the case of the transmitting antenna or, conversely, to effectively absorb the incident 
power from that direction as in the case of the receiving antenna, is specified in terms of 
its directive gain or directivity. For a direction specified by the angle pair (6, $), the directive 
gain of an antenna, denoted by g(6, $) is defined as the ratio of the radiation intensity in 
that direction to the average radiated power, as shown by 

(0, >) 


g, 6) =p 


_ D0, >) 
Plán 


The directivity of an antenna, denoted by D, is defined as the ratio of the maximum 
radiation intensity from the antenna to the radiation intensity from an isotropic source. 
That is, the directivity D is the maximum value of the directive gain g(0, p). Thus, whereas 
the directive gain of the antenna is a function of the angle pair (8, $), the directivity D is 
a constant that has been maximized for a particular direction. 

The definition of directivity is based on the shape of the radiation-intensity pattem 
(6, $); as such, it does not involve the effect of antenna imperfections due to dissipation 
loss and impedance mismatch. A quantity called power gain does involve the radiation 
efficiency of the antenna. Specifically, the power gain of an antenna, denoted by G, is 
defined as the ratio of the maximum radiation intensity from the antenna to the radiation 
intensity from a lossless isotropic source, under the constraint that the same input power 
is applied to both antennas. Specifically, using T.adiation tO denote the radiation efficiency 
factor of the antenna, we may relate the power gain G to the directivity D as 


G = Mradiation (8.9 ) 


Thus, the power gain of an antenna over a lossless isotropic source equals the directivity 
if the antenna is 100 percent efficient {i.e., radiation = 1), but it is less than the directivity. 


(8.8) 





8.4 Radio Link Analysis 521 


if any losses are present in the antenna (i-e., Y,adiation < 1). Henceforth, we assume that the 
antenna is 100 percent efficient and therefore refer only to the power gain of the antenna. 

The concept of power gain, which is based on the transmitted power-pattern shape, 
can be extended to a receiving antenna by virtue of the reciprocity principle. An antenna 
is said to be reciprocal if the transmission medium is linear, passive and isotropic. For a 
given antenna structure, the power gains of transmitting and receiving antennas are then 
identical. 

The power gain of an antenna is the result of concentrating the power density in a 
restricted region smaller than 47 steradians, as illustrated in Figure 8.6. In light of the 
picture portrayed in this figure, we may introduce the following two parameters: 


1. Effective radiated power referenced to an isotropic source (EIRP); the EIRP is defined 
as the product of the transmitted power, P,, and the power gain of the transmitting 
antenna, G,, as shown by 


EIRP = P,G, watts (8.10) 


2. Antenna beamwidth, representing a “planar” measure of the antenna’s solid angle 
of view; the beamwidth, in degrees or radians, is defined as the angle that subtends 
` the two points on the mainlobe of the field-power pattern at which the peak field 
power is reduced by 3 dBs. The higher the power gain of the antenna, the narrower 

is the antenna beamwidth. 


Another matter of interest discernible from Figure 8.6 is the sidelobes of the field-power 
pattern. Unfortunately, every physical antenna has sidelobes, which are responsible for 
absorbing unwanted interfering radiations. 


Effective Aperture 


A term that has a special significance for a receiving antenna is the effective aperture 
of the antenna, which is defined as the ratio of the power available at the antenna terminals 
to the power per unit area of the appropriately polarized incident electromagnetic wave. 
The effective aperture, denoted by A, is defined in terms of the antenna’s power gain G as 


A2 
ASG (8.11) 


where A is the wavelength of the carrier. The wavelength A and frequency f are reciprocally 
related as 


à=- (8.12) 


where c is the speed of light (approximately equal to 3 X 10° m/s). 
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Transmitting [4 Antenna Point of peak 
antenna x beamwidth output power 





Side lobes Points where the output power 
is 3 dBs below its peak value 


FIGURE 8.6 Illustrating the concentration of power density of a transmitting antenna inside a 
region smaller than 47 steradians. 
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The term effective aperture has particular significance in the context of reflector 
antennas and electromagnetic horns that are characterized by a well-defined aperture. For 
these antennas, the ratio of the antenna’s effective aperture to its physical aperture is q 
direct measure of the antenna’s aperture efficiency, Napertures iN radiating power to a desired 
direction or absorbing power from that direction. Nominal values for the efficiency 
aperture Of reflector antennas lie in the range of 45 to 75 percent. 


Friis Free-Space Equation 


With this introductory material on antennas at hand, we are now ready to formulate 
the basic propagation equation for a radio communication link. Consider a transmitting 
antenna with an EIRP defined in Equation (8.10). Invoking the inverse-square law 
of Equation (8.3), we may express the power density of the transmitting antenna as 
EIRP/47d?, where d is the distance between the receiving and transmitting antennas. The 
power P, absorbed by the receiving antenna is the product of this power density and the 
antenna’s effective area denoted by A,, as shown by 
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According to the reciprocity principle, we may use Equation (8.11) to express the effective 
area of the receiving antenna as 


A= 
"An 
where G, is the power gain of the receiving antenna. Substituting this formula for A, into 
Equation (8.13), we may express the received signal power in the equivalent form 


ay 
P, = recz) (8.14) 
Equation (8.14) is called the Friis free-space equation.“ 
The path loss, PL, representing signal “attenuation” in decibels across the entire 
communication link, is defined as the difference (in decibels) between the transmitted signal 
power P, and received signal power P,, as shown by 
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The minus sign associated with the first term in Equation (8.15) signifies the fact that this 
term represents a “gain.” The second term, due to the collection of terms (4md/A)”, is called 
the free-space loss, denoted by Lice space- Note that increasing the distance d separating 
the receiving antenna from the transmitting antenna causes the free-space loss to increase; 
which, in turn, compels us to operate the radio communication link at lower frequencies 
so as to maintain the path loss at a manageable level. 

The Friis free-space equation enables us to calculate the path loss PL for specified 
values of power gains G, and G,, the carrier wavelength A, and distance d. To complete 
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the budget link analysis, we need to calculate the average noise power in the received 
signal, which is considered next. 


= NOISE FIGURE 


To perform noise analysis at the receiver of a communication system, we need a convenient 
measure of the noise performance of a linear two-port device. One such measure is fur- 
nished by the so-called noise figure. Consider a linear two-port device connected to a signal 
source of internal impedance Z(f) = R(f) + jX(f) at the input, as in Figure 8.7. The noise 
voltage v(t) represents the thermal noise associated with the internal resistance R(f) of the 
source. The output noise of the device is made up of two contributions, one due to the 
source and the other due to the device itself. We define the available output noise power 
in a band of width Af centered at frequency f as the maximum average noise power in 
this band, obtainable at the output of the device. The maximum noise power that the two- 
port device can deliver to an external load is obtained when the load impedance is the 
complex conjugate of the output impedance of the device, that is, when the resistance is 
matched and the reactance is tuned out. We define the noise figure of the two-port device 
as the ratio of the total available output noise power (due to the device and the source) 
per unit bandwidth to the portion thereof due solely to the source. 

Let the spectral density of the total available noise power of the device output be 
Sno(f), and the spectral density of the available noise power due to the source at the device 
input be Sys(f). Also let G(f) denote the available power gain of the two-port device, 
defined as the ratio of the available signal power at the output of the device to the available 
signal power of the source when the signal is a sinusoidal wave of frequency f. Then we 
may express the noise figure F of the device as 


___Syolf) 
F = Snl) (8:16) 


If the device were noise free, Snolf) = G(f)Sns(f), and the noise figure would then be 
unity. In a physical device, however, Swo(f) is larger than G(f)Sys(f), so that the noise 
figure is always larger than unity. The noise figure is commonly expressed in decibels, that 
is, as 10 logio F. 

The noise figure may also be expressed in an alternative form. Let Ps(f) denote the 
available signal power from the source, which is the maximum average signal power that 
can be obtained. For the case of a source providing a single-frequency signal component 
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FIGURE 8.7 Linear two-port device. 
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with open-circuit voltage Vo cos(27ft), the available signal power is obtained when the 
load connected to the source is 


Z*(f) = R(F) — 1X(f) 


where the asterisk denotes complex conjugation. Under this condition, we find that 
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ARIF) 
The available signal power at the output of the device is therefore 
Polf) = G(f)Ps(f) (8.18) 


Then, multiplying both the numerator and denominator of the right-hand side of Equation 
(8.16) by Ps(f) A(f), we obtain 
Ps(f )Snolf) Af 
G(f)Ps(f)Sus(f) Af 
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where 
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We refer to ps(f) as the available signal-to-noise ratio of the source and to po(f) as the 
available signal-to-noise ratio at the device output, both measured in a narrow band of 
width Af centered at f. Since the noise figure is always greater than unity, it follows from 
Equation (8.19) that the signal-to-noise ratio always decreases with amplification, which 
is a significant result. 

The noise figure F is a function of the operating frequency f; it is therefore referred 
to as the spot noise figure. In contrast, we may define an average noise figure Fy of a two- 
port device as the ratio of the total noise power at the device output to the output noise 
power due solely to the source. That is, 


ps(f) (8.20) 
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fi cunsath af 
It is apparent that in the case of thermal noise in the input circuit with R(f) constant and 
constant gain throughout a fixed band with zero gain at other frequencies, the spot noise 
figure F and the average noise figure F, are identical. 
Equivalent Noise Temperature 


A disadvantage of the noise figure F is that when it is used to compare low-noise 
devices, the values obtained are all close to unity, which makes the comparison rather 
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FIGURE 8.8 Linear two-port device matched to the internal resistance of a source connected to 
the input. 


difficult. In such cases, it is preferable to use the equivalent noise temperature. Consider a 
linear two-port device whose input resistance is matched to the internal resistance of the 
source as shown in Figure 8.8. In this diagram, we have also included the noise voltage 
generator associated with the internal resistance R, of the source. The mean-square value 
of this noise voltage is 4kTR, Af, where k is Boltzmann’s constant. Hence, the available 
noise power at the device input is 


N, = kT Af (8.23) 


Let N; denote the noise power contributed by the two-port device to the total available 
output noise power N3. We define N; as 


N, = GRT, Af (8.24) 


where G is the available power gain of the device and T, is its equivalent noise temperature. 
Then it follows that the total output noise power is 


Nz = GN, + N; 








2 
= Gk(T + T.) Af (8:23) 
The noise figure of the device is therefore (see the output port of Figure 8.8) 
F Mos ee (8.26) 


N, - N; T 
Solving for the equivalent noise temperature: 
T. = T(F - 1) (8.27) 


The noise figure F is measured under matched input conditions, and with the noise source 
at temperature T. By convention the temperature T is taken as “room temperature,” 
namely 290 K, where K stands for “degree Kelvin.” 


Cascade Connection of Two-Port Networks 


It is often necessary to evaluate the noise figure of a cascade connection of two-port 
networks whose individual noise figures are known. Consider Figure 8.9, consisting of a 
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FIGURE 8.9 A cascade of two noisy two-port networks, 


pair of two-port networks of noise figures F; and F, and power gains G, and G3, connected 
in cascade. It is assumed that the devices are matched, and that the noise figure F, of the 
second network is defined assuming an input noise power N4. 

At the input of the first network, we have a noise power N; contributed by the source, 
plus an equivalent noise power (F, — 1)N, contributed by the network itself. The output 
noise power from the first network is therefore F,N,;G,. Added to this noise power at the 
input of the second network, we have the equivalent extra power (Fz — 1)N, contributed 
by the second network itself. The output noise power from this second network is therefore 
equal to F,G;NiG2 + (F — 1)NiG2. We may consider the noise figure F as the ratio of 
the actual output noise power to the output noise power assuming the networks to be 
noiseless, We may therefore express the overall noise figure of the cascade connection of 
Figure 8.9 as 


_ F,G,N,G, + (F vi 1)N,G, 
7 N,G,G2 
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The result may be readily extended to the cascade connection of any number of two-port 
networks, as shown by 
F,-1 5-1 Fy-1 


+ + +: ; 
Gı G,G,  G,G,G; (8.23) 
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where F}, Fa, F;, . . . are the individual noise figures, and G}, G2, G3, . - . are the available 
power gains, respectively. Equation (8.29) shows that if the first stage of the cascade 
connection in Figure 8.9 has a high gain, the overall noise figure F is dominated by the 
noise figure of the first stage. 
Correspondingly, we may express the overall equivalent noise temperature of the 
cascade connection of any number of noisy two-port networks as follows: 
T2 T; Ts 


T. = T, +¢24+—-4+- 4: 
e 1! Gi GGe GGG; 





(8.30) 


where Ti, Tz, T3,..- are the equivalent noise temperatures of the individual networks, 
and Gi, G2, G3, . . . are the available power gains, respectively. Equation (8.30) is known. 
as the Friis formula. Here again we note that if the gain G; of the first stage is high, the 
equivalent noise temperature T, is dominated by that of the first stage. 


P EXAMPLE 8.1 Noise Temperature of Earth-Terminal Receiver 


Figure 8.10 shows a typical earth-terminal receiver, consisting of a low-noise radio-frequency 
(RF) amplifier (LNA), frequency down-converter (mixer), and intermediate frequency (IF) 
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Ficure 8.10 Block diagram of earth terminal receiver. 


amplifier. The equivalent noise temperatures of these components, including the receiving 
antenna, are 


Teotenna = 50 K 
Ter = 50K 
Tmiser = 500 K 

Tr = 1000 K 


The available power gains of the two amplifiers are 
Grr = 200 = 23 dB 
Gp = 1000 = 30 dB 


To calculate the equivalent noise temperature of the receiver, we use Equation (8.30), 
obtaining 
Tia F 
T, = Teen + Tae + mixer Tr 
Grr 
500 + 1000 
200 


= 107.5 K <q 


= 50 + 50 + 


> EXAMPLE 8.2 Downlink Budget Analysis of a Digital Satellite 
Communication System 


In a digital satellite communication system, one of the key elements in the overall design and 
analysis of the system is the downlink power budget, which is usually more critical than the 
uplink power budget because of the practical constraints imposed on downlink power and 
satellite antenna size. The example presented here addresses a sample downlink budget anal- 
ysis, assuming that any required uplink power (within limits) is available for satisfactory 
operation of the system. 

The critical parameter to be calculated is the ratio of received carrier power-to-noise 
spectral density, denoted by C/No. According to the Friis free-space equation (8.14), the av- 
erage power received at the earth terminal to the average power P, transmitted by the satellite 


is 
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where, in this example, G, is the power gain of the satellite antenna, G, is the power gain of 
the receiving earth-terminal antenna, A is the carrier wavelength for the downlink, and q is 
the distance between the satellite and the earth terminal. Given that the equivalent noise 
temperature of the system is T,, we may use Equation (1.94) of Chapter 1 to express the noise 
spectral density No as kT., where k is Boltzmann’s constant. Moreover, from Equation (8.10) 
we note that P,G, is equal to the EIRP of the satellite. Hence, dividing P, by No, we May 
express the C/N, ratio for the downlink as 


2 
c EIRP pete & Ay. 
(a = hae) ales) k (8.31) 


For a given satellite system, the free-space loss (47d/A)* is a constant. Viewing the system from 
the earth terminal, we see from Equation (8.31) that the (C/No) ratio is proportional to 
G,/T,. The ratio G,/T, may therefore be used to assess the “quality” of an earth terminal; it 
is usually shortened to the G/T ratio, which is referred to as the figure of merit of the receiving 
earth terminal. Thus, rewriting the formula (8.31) for the (C/No) ratio measured in decibels, 
we may express it as the sum of gains and losses as itemized here: 


1. (EIRP) cstetties Measured in dBW, where dBW denotes decibels referenced to 1 watt, that 
is, 0 dBW. 

2. (G/T)earth terminals Measured in dB/K, where K refers to degree Kelvin. 

3. Lyre spaces denoting the free-space loss 10 logio(4ad/A)? in dB. 

4. —10 logio k, representing the gain in dBW/K-Hz due to division by the Boltzmann 
constant k = 1.38 X 10773 joule/K. 


Table 8.1 presents the values of these four terms for the downlink of a typical domestic digital 
satellite communication system, based on the following: 


1. The transponder is operated at its maximum output power (i.e., no power backoff is 
employed), yielding an EIRP of 46.5 dBW. 

The receiving earth terminal uses a 2m-dish antenna with a power gain G = 45 dB, and 
the receiver is configured as in Example 8.1 with equivalent temperature T = 107.5 K. 


Hence 


2 


S45 = 40 logig 107.5 


T 
= 45 — 20.3 
= 24.7 dB/K 
3. The free-space loss is 
Liwespace = 92.4 + 20 logio f + 20 logio d dB (8.32) 


TABLE 8.1 Downlink power 
budget for Example 8.2 





Variable Value 
EIRP +46.5 dBW 
GIT ratio +24.7 dB/K 
Free-space loss —206 dB 


Boltzmann constant +228.6 dBW/K-Hz 


CINo 


93.8 dB-Hz 
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where the downlink carrier frequency f is in GHz and the distance d between the satellite 
and the earth terminal is in kilometers. For a geostationary satellite, the distance between 
the satellite and an earth terminal lies in the range of 36,000 to 41,000 km. Thus 
choosing d = 40,000 km and assuming f = 12 GHz, the use of Equation (8.32) yields 


Lyroe-space = 92.4 + 20 logio 12 + 20 login 40,000 
= 92.4 + 21.6 + 92.0 
= 206 dB 


4. With the Boltzmann constant k = 1.39 X 107%” joule/K, its contribution to the C/No 
ratio is 


—10 logio k = 10 logo 1.38 X 10778 
= 228.6 dBW/K-Hz 


Totaling the gains and losses, we thus get 


C 
= = 93.8 dB-Hz 
(S) 


The “received” downlink value of the (C/No) ratio may also be expressed in terms of 
the “required” value of the bit energy-to-noise spectral density ratio, (E,/No)req dB, at the 
receiving earth terminal as (see Equation (8.2)) 

(£) z (=) + 10 logio M + 10 logo R dB (8.33) 

No /downlink No/req 
where 10 logyo M is the link margin in decibels, and R is the data rate in b/s. The link margin 
allows for excess rain losses in propagation and other power degradations. Typically, the link 
margin is selected as 4 dB for C-band, 6 dB for Ku-band, and higher for the higher K-band 
frequencies because of the higher rain losses. For operation at the Ku-band frequency of 
12 GHz, we choose a link margin of 6 dB. Thus, using the value C/No = 93.8 dB-Hz calculated 
from the link budget, the link margin 10 logiy M = 6 dB, and assuming (E,/No)req = 12.5 dB, 
the use of Equation (8.33) yields 


10 logio R = 93.8 — 12.5 — 6 
= 75.3 


Hence, 
R = 33.9 Mb/s 


Assuming the use of coherent 8-PSK for the transmission of digital data via the satellite, 
and substituting (E,/No) = 12.5 dB in Equation (6.47) of Chapter 6, we find that the prob- 
ability of symbol error P, = 0.6 xX 107°. 

To summarize, the digital satellite communication system analyzed in this example per-' 
mits, under the worst operating conditions, data transmission on the downlink at a rate 
R = 33,9 Mb/s and with a probability of symbol error P, = 0.6 X 10%, assuming the use of 
8-phase PSK. < 


i 8.5 Wireless Communications 


In this section we study the second type of multiuser radio communication system, namely, 
wireless communications, which is synonymous with mobile radio. The term mobile radio 
is usually meant to encompass indoor or outdoor forms of wireless communications where 
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a radio transmitter or receiver is capable of being moved, regardless of whether it actually 
moves or not. Due to the stochastic nature of the mobile radio channel, its characterization 
mandates the use of practical measurements and statistical analysis. The aim of such an 
evaluation is to quantify two factors of primary concern: 


1. Median signal strength, which enables us to predict the minimum power needed to 
radiate from the transmitter so as to provide an acceptable quality of coverage oyver 
a predetermined service area. 


2. Signal variability, which characterizes the fading nature of the channel. 


Our specific interest in wireless communications is in the context of cellular radio’ 
that has the inherent capability of building mobility into the telephone network. With such 
a capability, a user can move freely within a service area and simultaneously communicate 
with any telephone subscriber in the world. An idealized model of the cellular radio system, 
illustrated in Figure 8.11, consists of an array of hexagonal cells with a base station located 
at the center of each cell; a typical cell has a radius of 1 to 12 miles. The function of the 
base stations is to act as an interface between mobile subscribers and the cellular radio 
system. The base stations are themselves connected to a switching center by dedicated 
wirelines. 

The mobile switching center has two important roles. First, it acts as the interface 
between the cellular radio system and the public switched telephone network. Second, it 
performs overall supervision and control of the mobile communications. It performs the: 
latter function by monitoring the signal-to-noise ratio of a call in progress, as measured 
at the base station in communication with the mobile subscriber involved in the call. When 
the SNR falls below a prescribed threshold, which happens when the mobile subscriber 
leaves its cell or when the radio channel fades, it is switched to another base station. This 
switching process, called a handover or handoff, is designed to move a mobile subscriber 
from one base station to another during a call in a transparent fashion, that is, without 
interruption of service. 

The cellular concept relies on two essential features, as described here: 


1. Frequency reuse. The term frequency reuse refers to the use of radio channels on the 
same carrier frequency to cover different areas, which are physically separated from 


Cell 





Base station 














FiGure 8.11 Idealized model of cellular radio. 
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each other sufficiently to ensure that co-channel interference is not objectionable. 
Thus, instead of covering an entire local area from a single transmitter with high 
power at a high elevation, frequency reuse makes it possible to achieve two com- 
monsense objectives: keep the transmitted power from each base station to a mini- 
mum, and position the antennas of the base stations just high enough to provide for 
the area coverage of the respective cells. 

2. Cell splitting. When the demand for service exceeds the number of channels allocated 
to a particular cell, cell splitting is used to handle the additional growth in traffic 
within that particular cell. Specifically, cell splitting involves a revision of cell bound- 
aries, so that the local area formerly regarded as a single cell can now contain a 
number of smaller cells and use the channel complements of these new cells. The new 
cells, which have a smaller radius than the original cells, are called microcells. The 
transmitter power and the antenna height of the new base stations are correspond- 
ingly reduced, and the same set of frequencies are reused in accordance with a new 
plan. 


For a hexagonal model of the cellular radio system, we may exploit the basic prop- 
erties of hexagonal cellular geometry to lay out a radio channel assignment plan that 
determines which channel set should be assigned to which cell. We begin with two integers 
i andj (i = j), called shift parameters, which are predetermined in some manner. We note 
that with a hexagonal cellular geometry there are six “chains” of hexagons that emanate 
from each hexagon and that extend in different directions. Thus, starting with any cell as 
a reference, we find the nearest co-channel cells by proceeding as follows: 


>= Move i cells along any chain of hexagons, turn counterclockwise 60 degrees, and 
move j cells along the chain that lies on this new direction. The jth cells so located 
and the reference cell constitute the set of co-channel cells. 


This procedure is repeated for a different reference cell, until all the cells in the system are 
covered. Figure 8.12 illustrates the application of this procedure for a single reference cell 
and the example of i = 2 and; = 2. 

In North America, the band of radio frequencies assigned to the cellular system is 
800-900 MHz. The subband 824-849 MHz is used to receive signals from the mobile 
units, and the subband 869-894 MHz is used to transmit signals to the mobile units. The 




















Ficure 8.12 Illustrating the determination of co-channel cells. 
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use of these relatively high frequencies has the beneficial feature of providing a good por- 
table coverage by penetrating buildings. In Europe and elsewhere, the base—mobile ang 
mobile—base subbands are reversed. 5 


a PROPAGATION EFFECTS® 


The major propagation problems encountered in the use of cellular radio in built-up areas 
are due to the fact that the antenna of a mobile unit may lie well below the surrounding 
buildings. Simply put, there is no “line-of-sight” path to the base station. Instead, radio 
propagation takes place mainly by way of scattering from the surfaces of the surrounding 
buildings and by diffraction over and/or around them, as illustrated in Figure 8.13. The 
important point to note from Figure 8.13 is that energy reaches the receiving antenna via 
more than one path. Accordingly, we speak of a multipath phenomenon in that the various 
incoming radio waves reach their destination from different directions and with different 
time delays. 

To understand the nature of the multipath phenomenon, consider first a “‘static” 
multipath environment involving a stationary receiver and a transmitted signal that con- 
sists of a narrowband signal (e.g., unmodulated sinusoidal carrier). Let it be assumed that 
two attenuated versions of the transmitted signal arrive sequentially at the receiver. The 
effect of the differential time delay is to introduce a relative phase shift between the two 
components of the received signal. We may then identify one of two extreme cases that 
can arise: 


» The relative phase shift is zero, in which case the two components add constructively, 
as illustrated in Figure 8.14a. 

» The relative phase shift is 180 degrees, in which case the two component add de- 
structively, as illustrated in Figure 8.14. 


We may also use phasors to demonstrate the constructive and destructive effects of mul- 
tipath, as shown in Figures 8.154 and 8.15, respectively. Note that in the static multipath 
environment described herein, the amplitude of the received signal does not vary with time. 

Consider next a “dynamic” multipath environment in which the receiver is in motion 
and two versions of the transmitted narrowband signal reach the receiver via paths of 







Direction ao 
to elevated 
base station Tree: 
Obstructed = 
line-of-sight 


path % 


Building 


FiGure 8.13 Illustrating the mechanism of radio propagation in urban areas. (From Parsons, 
1992, with permission.) i 
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FIGURE 8.14 (a) Constructive and (b) destructive forms of the multipath phenomenon for sinu- 
soidal signals. 


different lengths. Due to motion of the receiver, there is a continuous change in the length 
of each propagation path. Hence, the relative phase shift between the two components of 
the received signal is a function of spatial location of the receiver. As the receiver moves, 
we now find that the received amplitude (envelope) is no longer constant as was the case 
in a static environment; rather, it varies with distance, as illustrated in Figure 8.16. At the 
top of this figure, we have also included the phasor relationships for the two components 
of the received signal at various locations of the receiver. Figure 8.16 shows that there is 
constructive addition at some locations, and almost complete cancellation at some other 
locations. This phenomenon is referred to as signal fading. 

In a mobile radio environment encountered in practice, there may of course be a 
multitude of propagation paths with different lengths, and their contributions to the re- 
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FIGURE 8.15 Phasor representations of (a) constructive and (b) destructive forms of multipath. 
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FIGURE 8.16 Illustrating how the envelope fades as two incoming signals combine with differ- 
ent phases. (From Parsons, 1992, with permission.) 


ceived signal could combine in a variety of ways. The net result is that the envelope of the 
received signal varies with location in a complicated fashion, as shown by the experimental 
record of received signal envelope in an urban area that is presented in Figure 8.17. This 
figure clearly displays the fading nature of the received signal. The received signal envelope 
in Figure 8.17 is measured in dBm. The unit dBm is defined as 10 logio(P/P5), with P 
denoting the power being measured and Po = 1 milliwatt. In the case of Figure 8.17, P is 
the instantaneous power in the received signal envelope. 

Signal fading is essentially a spatial phenomenon that manifests itself in the time 
domain as the receiver moves. These variations can be related to the motion of the receiver 
as follows. To be specific, consider the situation illustrated in Figure 8.18, where the re- 
ceiver is assumed to be moving along the line AA’ with a constant velocity v. It is also 
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FIGURE 8.17 Experimental record of received signal envelope in an urban area. (From Parsons, 
1992, with permission.) 
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FIGURE 8.18 Illustrating the calculation of Doppler shift. 


assumed that the received signal is due to a radio wave from a scatterer labeled S. Let At 
denote the time taken for the receiver to move from point A to A’, Using the notation 
described in Figure 8.18, the incremental change in the path length of the radio wave is 


deduced to be 


Al = d cosa 


(8.34) 
=-—vAtcosa 


where a is the spatial angle between the incoming radio wave and the direction of motion 
of the receiver. Correspondingly, the change in the phase angle of the received signal at 
point A’ with respect to that at point A is given by 


ag = 77 aI 


8.35 
_2av At ( ) 


À 





> cos@ 


where À is the radio wavelength. The apparent change in frequency, or the Doppler-shift, 
is therefore 


1 Ad 


2r At 


(8.36) 


v 
= — cosa 
À 


The Doppler-shift vis positive (resulting in an increase in frequency) when the radio waves 
arrive from ahead of the mobile unit, and it is negative when the radio waves arrive from 
behind the mobile unit. 


8.6 Statistical Characterization 
of Multipath Channels 


The narrowband characterization of the multipath environment described in Section 8.5 
is appropriate for mobile radio transmissions where the signal bandwidth is very small 
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compared to the reciprocal of the spread in propagation path delays. Multipath in such 
an environment results in two effects: rapid fading of the received signal envelope and 4 
spread in Doppler shifts in the received spectrum. Real-life signals radiated in a mobile 
radio environment may, however, occupy a bandwidth wide enough to require more de. 
tailed considerations of the effects of multipath propagation on the received signal. In this 
section, we present a statistical characterization of a mobile radio channel.* 

Consider a mobile radio channel with multiple propagation paths. In accordance 
with the complex notation described in Appendix 2, we may express the transmitted band. 
pass signal as 


s(t) = Re[S(t) exp(j27f.t)] (8.37) 


where &(t) is the complex (low-pass) envelope of s(t), and f is a nominal carrier frequency, 
Since the channel is time varying due to multipath effects, the impulse response of the 
channel is delay dependent and therefore a time-varying function. Let the impulse response 
of the channel be expressed as 


h( rst) = Relh(r3t) expl j27f t) (8.38) 


where h(7;¢) is the (low-pass) complex impulse response of the channel, and 7 is a delay 
variable. The complex impulse response Dirt) is called the input delay-spread function of 
the channel. The (low-pass) complex envelope of the channel output is defined by the 
convolution integral . 

3 1{* . 

§,(t) = eae a(t — 7)b(r3t) dt (8.39) 


where the scaling factor } is the result of using complex notation. 

In genéral;the behavior of a mobile radio channel can be described only in statistical 
terms. For analytic purposes, the delay-spread function h(7;t) may thus be modeled as a 
zero-mean complex-valued Gaussian process. Then, at any time ż the envelope | (7;¢)]| is 
Rayleigh distributed, and the channel is referred to as a Rayleigh fading channel. When, 
however, the mobile radio environment includes fixed scatterers, we are no longer justified 
in using a zero-mean model to describe the input delay-spread function h(73t). In such a 
case, it is more appropriate to use a Rician distribution to describe the envelope | h(73t)|, 
and the channel is referred to as a Rician fading channel. The Rayleigh and Rician distri- 
butions for a real-valued random process were considered in Chapter 1. In the discussion 
presented in this chapter, we consider only a Rayleigh fading channel. 

The time-varying transfer function of the channel is defined as the Fourier transform 
of the input delay-spread function h(7;t) with respect to the delay variable 7, as shown by 


Alfst) = i A(r3t) exp(—j2af 7) dr (8.40) 


where f denotes the frequency variable. The time-varying transfer function Hi(f;t) may be 
viewed as a frequency transmission characteristic of the channel. 


*Readers who are not interested in the mathematical details pertaining to the statistical characterization of fading 
multipath channels, may skip the material presented in this section, except for the subsection on the classification 
of multipath channels at the end of the section. 
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For a statistical characterization of the channel, we make the following assumptions: 


> The input delay-spread function A(1;t) is a zero-mean, complex-valued Gaussian 
process. Our interest is confined to short-term fading; it is therefore reasonable to 
assume that A(7;t) is also stationary. Because Fourier transformation is linear, the 
time-varying transfer function Ĥ(f;t) has similar statistics. 

» The channel is an uncorrelated scattering channel, which means that contributions 
from scatterers with different propagation delays are uncorrelated. 


Consider then the autocorrelation function of the input delay-spread function /(7;t). Since 
h(7;t) is complex valued, we use the following definition for the autocorrelation function: 


RiTo Tab) = E[h* (m;t) (Tast) (8.41) 


where E is the statistical expectation operator, the asterisk denotes complex conjugation, 
7, and 7, are the propagation delays of the two paths involved in the calculation, and ¢, 
and ż, are the times at which the outputs of the two paths are observed. Invoking station- 
arity in the time variable ¢ and uncorrelated scattering in the time-delay variable 7, we may 
reformulate the autocorrelation function of f(7;t) as 


E[h* (rt) birt + At)] 
77,(4%43At) ôT — 7) 


Rj(t4,72;At) 


(8.42) 


where At is the difference between the observation times, and 5(7, — 72) is a delta function. 
Using 7 in place of 7, the remaining function in Equation (8.42) is redefined as 


rg(t;At) = E[h(23t)h*(73¢ + At)] (8.43) 


The function rj(7;Az) is called the multipath autocorrelation profile of the channel. 

Consider next a statistical characterization of the channel in terms of the complex- 
valued, time-varying transfer function H(f;¢). Following a formulation similar to that de- 
scribed in Equation (8.41), the autocorrelation function of H(f;t) is defined by 


Ralfistisfaste) = ELA (fasti)A( fost) (8.44) 


where f, and fz represent two frequencies in the spectrum of a transmitted signal. The 
autocorrelation function Ra(f1,t1;f2;t2) provides a statistical measure of the extent 
to which the signal is distorted by transmission through the channel. From Equations 
(8.40), (8.41), and (8.44) we find that the autocorrelation functions Ra(f1,t13f2,t2) and 
Rj (715137252) are related by a form of two-dimensional Fourier transformation as follows: 


Ralfistufost2) = i T RE(Tisti Tosta) exp[j2a(fit, — fot)] dra dra (8.45) 
Invoking stationarity in the time domain, we may reformulate Equation (8.44) as 
RalfifosAt) = ELA*(fist)Alfast + At) (8.46) 


This definition suggests that the autocorrelation function Ra(f1,f2;At) may be measured 
by pairs of spaced tones to carry out cross-correlation measurements on the resulting 
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channel outputs. Such a measurement presumes stationarity in the time domain. If we also 
assume stationarity in the frequency domain, we may go one step further and write 


Ralf,f + Af;At) 


rp(Af;At) 
EIR (SOES + Afst + Ad] (8.47) 


This specialized form of the autocorrelation function of H(f;t) is in fact the Fourier trans- 
form of the multipath autocorrelation profile 7j(7,4t) with respect to the delay-time vari. 
able 7, as shown by 


TITA) exp(—j2ar Af) dr (8.48) 


ral fide) = | 
The function rp(Af;At) is called the spaced-frequency spaced-time correlation function of 
the channel. i 
Finally, we introduce a function S(r7;v) that forms a Fourier-transform pair with the 
multipath autocorrelation profile 7;(7;At) with respect to the variable At, as shown by 


S(73v) = a 7(7;At) exp(—j2av At) d(At) (8.49) 
and 
bo = iQ S(t3v) exp(j2av At) dv (8.50) 


\ 


The function S(7;v) may also be defined in terms of 77,(Af;At) by applying a form of double 
Fourier transformation: a Fourier transform with respect to the time variable At and an 
inverse Fourier transform with respect to the frequency variable Af. That is to say, 


S(73v) = ie T ralAf;At) exp(—j2mv At) exp(j2m7 Af) d(At) d(Af) (8.51) 


Figure 8.19 displays the functional relationships between 7j(7;At), ralAf;At), and S(7;») 
in terms of the Fourier transform and its inverse. 
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FIGURE 8.19 Functional relationships between the multipath autocorrelation profile rj(7;At), the 
spaced-frequency spaced-time correlation function rq(Af;At), and the scattering function $(7;¥). 
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The function S(7;v) is called the scattering function of the channel. For a physical 
interpretation of it, consider the transmission of a single tone of frequency f’ (relative to 
the carrier). The complex envelope of the resulting filter output is 


§,(t) = expl j2mf’t)A(f'st) (8.52) 
The autocorrelation function of §,(f) is 


EJES (HSol + At)| = exp(j2af’ At)ELA*(f’st)A(f'st + At) 


= exp(j2af’ At)rq(0;At) (8-53) 


where, in the last line, we have made use of Equation (8.47). Putting Af = 0 in Equation 
(8.48), and then using Equation (8.50), we may write 


ral0;At) = E ri(7At) dr 
= [re (8.54) 
= [ [f S(T; v) ar| exp(j2av At) dy 


Hence, we may view the integral 


F Sir;v) dr 


as the power spectral density of the channel output relative to the frequency f’ of the 
transmitted tone, and with the Doppler shift v acting as the frequency variable. General- 
izing this result, we may state that the scattering function S(7;v) provides a statistical 
measure of the output power of the channel, expressed as a function of the time delay 7 
and the Doppler shift v. 


DELAY SPREAD AND DOPPLER SPREAD 
Putting At = 0 in Equation (8.43), we may write 


P5(7) = 75(730) 


= E[|Blest)|*I men) 


The function P;(r) describes the intensity (averaged over the fading fluctuations) of the 
scattering process at propagation delay 7. Accordingly, P(r) is called the delay power 
spectrum or the multipath intensity profile of the channel. The delay power spectrum may 
also be defined in terms of the scattering function S(7;v) by averaging it over all Doppler 
shifts. Specifically, putting At = 0 in Equation (8.50) and then using the first line of Equa- 
tion (8.55), we may write 


P;(7) = n Slr;v) dv (8.56) 


Figure 8.20 shows an example of a delay power spectrum that depicts a typical plot 
of the power spectral density versus excess delay; the excess delay is measured with respect 
to the time delay for the shortest echo path. Note, as in Figure 8.17, the power is measured 
in dBm. The “threshold level” included in Figure 8.20 defines the power level below which 
the receiver fails to operate satisfactorily. 
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FIGURE 8.20 Example of a power-delay profile for a mobile radio channel. (From Parsons, 
1992, with permission.) 


Two statistical moments of P;(7) of interest are the average delay, Tav, and the delay 
spread, c,. The average delay is defined as the first central moment (i.e., the mean) of P;(7), 
as shown by 


I 7P;,(7) dr 


0 


acai > a 
Í P3(7) dr 


The delay spread is defined as the square root of the second central moment of P;(7), as 
shown by 


(8.57) 


/2 


[, (= Pita) dr 





(8.58) 


Or 


X Piir) dr 


The reciprocal of the delay spread c, is a measure of the coherence bandwidth of the 
channel, which is denoted by B.. 

Consider next the issue of relating the Doppler effects to time variations of the chan- 
nel. For this purpose, we first set Af = 0, which corresponds to the transmission of a single 
tone (of some appropriate frequency) over the channel. The spaced-frequency spaced-time 
correlation function of the channel then reduces to r;(0;At). Hence, evaluating the Fourier 
transform of this function with respect to the time variable At, we may write 


Sal) = F ral0;At) exp(—}j2rv At) d(At) (8.59) 


The function Sg(v) defines the power spectrum of the channel output expressed as a func- 
tion of the Doppler shift v; it is therefore called the Doppler spectrum of the channel. The 
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Doppler spectrum may also be defined in terms of the scattering function by averaging it 
over all possible propagation delays, as shown by 


Sav) = W S(T;v) dr (8.60) 


The Doppler shift v may assume positive and negative values with equal likelihood. The 
mean Doppler shift is therefore zero. The square root of the second moment of the Doppler 
spectrum is thus defined by 

1/2 


T VSa(v) dv 
= | =5— (8.61) 


T, 


7 f Salv) dv 


The parameter g, provides a measure of the width of the Doppler spectrum; it is therefore 
called the Doppler spread of the channel. The reciprocal of the Doppler spread is called 
the coherence time of the channel, which is denoted by 7,. 

‘Another useful parameter that is often used in measurements is the fade rate of the 
channel. For a Rayleigh fading channel, the average fade rate is related to the Doppler 
spread o, as 


fe = 1.4750, crossings per second (8.62) 


As the name implies, the fade rate provides a measure of the rapidity of fading of the 
channel. 
Some typical values encountered in a mobile radio environment are as follows: 


» The delay spread, o,, amounts to about 20 ps. 
» The Doppler spread, o,, due to the motion of a vehicle may extend up to 40-80 Hz. 


m CLASSIFICATION OF MULTIPATH CHANNELS 


The particular form of fading experienced by a multipath channel depends on whether the 
channel characterization is viewed in the frequency domain or the time domain. 

When the channel is viewed in the frequency domain, the parameter of concern is 
the channel’s coherence bandwidth, B., which is a measure of the transmission bandwidth 
for which signal distortion across the channel becomes noticeable. A multipath channel is 
said to be frequency selective if the coherence bandwidth of the channel is small compared 
to the bandwidth of the transmitted signal. In such a situation, the channel has a filtering 
effect in that two sinusoidal components, with a frequency separation greater than the 
channel’s coherence bandwidth, are treated differently. If, however, the coherence band- 
width of the channel is large compared to the message bandwidth, the fading is said to be 
frequency nonselective, or frequency flat. 

When the channel is viewed in the time domain, the parameter of concern is the 
coherence time, T,, which provides a measure of the transmitted signal duration for which 
distortion across the channel becomes noticeable. The fading is said to be time selective if 
the coherence time of the channel is small compared to the duration of the received signal 
(i.e., the time for which the signal is in flight). For digital transmission, the received signal’s 
duration is taken as the symbol duration plus the channel’s delay spread. If, however, the 
channel’s coherence time is large compared to the received signal duration, the fading is 
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FIGURE 8.21 Illustrating the four classes of multipath channels: 7, = coherence time, 
B. = coherence bandwidth. 


said to be time nonselective, or time flat, in the sense that the channel appears to the 
transmitted signal as time invariant. 
In light of this discussion, we may classify multipath channels as follows: 


> Flat-flat channel, which is flat in both frequency and time. 
> Frequency-flat channel, which is flat in frequency only. 
> Time-flat channel, which is flat in time only. 
» Nonflat channel, which is flat neither in frequency nor in time; such a channel is 
sometimes referred to as a doubly dispersive channel. 
The classification of multipath channels, based on this approach, is shown in Figure 8.21. 


The forbidden area, shown shaded in this figure, follows from the inverse relationship that 
exists between bandwidth and time duration. 


8.7 Binary Signaling over a 
Rayleigh Fading Channel 





In Chapter 6, we determined the average probability of symbol error for the transmission 
of binary data over a channel corrupted by additive white Gaussian noise. In a mobile 
radio environment, we have an additional effect to consider, namely, the fluctuations in 
the amplitude and phase of the received signal due to multipath effects. To be specific, 
consider the transmission of binary data over a Rayleigh fading channel, for which the 
(low-pass) complex envelope of the received signal is modeled as follows: 


X(t) = æ exp(—j)s(t) + w(t) (8.63) 


where &(z) is the complex envelope of the transmitted (band-pass) signal, œ is a Rayleigh- 
distributed random variable describing the attenuation in transmission, @ is a uniformly 
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distributed random variable describing the phase-shift in transmission, and (t) is a com- 
plex-valued white Gaussian noise process. It is assumed that the channel is flat in both 
time and frequency, so that we can estimate the phase-shift @ from the received signal 
without error. Suppose then that coherent binary phase-shift keying is used to do the data 
transmission. Under the condition that a is fixed or constant over a bit interval, we may 
adapt Equation (6.20) of Chapter 6 for the situation at hand by expressing the average 
probability of symbol error (i.e., bit error rate) due to the additive white Gaussian noise 
acting alone as follows: 


PAY) = 3 erfe( Vy) (8.64) 


where yis an attenuated version of the transmitted signal energy per bit-to-noise spectral 
density ratio E,/No, as shown by 


yati (8.65) 


Now, insofar as a mobile radio channel is concerned, we may view P,(y) as a conditional 
probability given that a is fixed. Thus, to evaluate the average probability of symbol error 
in the combined presence of fading and noise, we must average P,(y) over all possible 
values of y, as shown by 


P, = [7 PANAN dy (8.66) 


where f(y) is the probability density function of y. From Equation (8.65) we note that y 
depends on the squared value of œ. Since œ is Rayleigh distributed, we find that y has a 
chi-square distribution with two degrees of freedom.’ In particular, we may express the 
probability density function of y as 


1 y ) 
= — exp| -—], 20 8.67) 
Fly) F of a Y ( 
The term y is the mean value of the received signal energy per bit-to-noise spectral density 
ratio, which is defined by 


Yo = Ely| 
E, (8.68) 


where E[a°] is the mean-square value of the Rayleigh-distributed random variable a. 
Substituting Equations (8.64) and (8.67) into (8.66), and carrying out the integration, we 


get the final result 
1f Yo ) 
P. =>5{1- | 8.69 
( 1+ 


Equation (8.69) defines the bit error rate for coherent binary phase-shift keying (PSK) 
over a flat-flat Rayleigh fading channel. Following a similar approach, we may derive the 
corresponding bit error rates for coherent binary frequency-shift keying (FSK), binary 
differential phase-shift keying (DPSK), and noncoherent binary FSK. The results of these 
evaluations are summarized in Table 8.2. In Figure 8.22, we have used the exact formulas 
of Table 8.2 to plot the bit error rate versus Yọ expressed in decibels. For the sake of 
comparison, we have also included in Figure 8.22 plots for the bit error rates of coherent 
binary PSK and noncoherent binary FSK for a nonfading channel. We see that Rayleigh 
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TABLE 8.2 Bit error rates for binary signaling over a flat-flat 
Rayleigh fading channel 


Approximate Formula 











Exact Formula for the for the Bit Error Rate, 
Type of Signaling Bit Error Rate P. Assuming Large Yo 

Coherent binary PSK T Yo å 
2 1+% 4% 

Coherent binary FSK 1 A Yo sle 
2 2+ y 2Y% 

Binary DPSK 1 1 
2(1 + y) 2Y 

Noncoherent binary FSK 1 1 
2+ Y% Yo 





fading results in a severe degradation in the noise performance of a digital passband trans- 
mission system, the degradation being measured in tens of decibels of additional mean 
signal-to-noise ratio compared to a nonfading channel for the same bit error rate. In par- 
ticular, for large yo we may derive the approximate formulas given in the last column of 
Table 8.2, according to which the asymptotic decrease in the bit error rate with the average 
signal energy per bit-to-noise spectral density ratio y follows an inverse law. This behavior 
is dramatically different from the case of a nonfading channel, for which the asymptotic 
decrease in the bit error rate with yo follows an exponential law. 

The practical implication of this difference is that in a mobile radio environment, we 
have to provide a large increase in mean signal-to-noise ratio (relative to a nonfading 
environment), so as to ensure a bit error rate that is low enough for practical use. To meet 
such a requirement, we have to increase the transmitted power, antenna size, and so on, 
which can be costly in terms of implementation. Alternatively, we may utilize special mod- 
ulation and reception techniques that are less vulnerable to fading effects. Among these 
techniques, the best known and most widely used are the multiple-receiver combining 
techniques referred to collectively as diversity, a brief discussion of which is presented 
next. 


DIVERSITY TECHNIQUES 


Diversity may be viewed as a form of redundancy. In particular, if several replicas of the 
message signal can be transmitted simultaneously over independently fading channels, then 
there is a good likelihood that at least one of the received signals will not be severely 
degraded by fading. There are several methods for making such a provision. In the context 
of our present discussion, the following diversity techniques are of particular interest: 


» Frequency diversity — 
> Time (signal-repetition) diversity 
> Space diversity 


In frequency diversity, the message signal is transmitted using several carriers that 
are spaced sufficiently apart form each other to provide independently fading versions 0. 
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FIGURE 8.22 Performance of binary signaling schemes over a Rayleigh fading channel, shown 
as continuous curves; the dashed curves pertain to a nonfading channel. 


the signal. This may be accomplished by choosing a frequency spacing equal to or larger 
than the coherence bandwidth of the channel. 

In time diversity, the same message signal is transmitted in different time slots, with 
the spacing between successive time slots being equal to or greater than the coherence time 
of the channel, Time diversity may be likened to the use of a repetition code for error- 
control coding. (Error-control coding is discussed in Chapter 10.) 

In space diversity, multiple transmitting or receiving antennas (or both) are used, 
with the spacing between adjacent antennas being chosen so as to assure the independence 
of fading events; this may be satisfied by spacing the adjacent antennas by at least seven 

times the radio wavelength. 

Given that by one of these means we create L independently fading channels, we 
may then use a linear diversity combining structure involving L separate receivers, as 
depicted in Figure 8.23. The system is designed to compensate only for short-term effects 
of a fading channel. Moreover, it is assumed that noise-free estimates of the channel at- 
tenuation factors {a,} and the channel phase-shifts {ġ;} are available. Then, the linear 
combiner achieves optimum performance for binary data transmission (discussed here for 
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Ficure 8.23 Block diagram illustrating the space diversity technique. 
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FIGURE 8.24 Performance of binary signaling schemes with diversity. (From Proakis, 1995, 
with permission of McGraw-Hill.) 
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the purpose of illustration) by proceeding as follows: The output of the kth matched filter 
in the fth receiver, ex(t), is multiplied by a, exp(jd,) that represents the complex conju- 
gate of the €th channel gain, where £ = 1, 2, . . . , L, and k = 0, 1. Thus, the linear combiner 
results in two output complex envelopes defined by 


L 
D(t) = 2 a EXp( jde) Talt),  k=0,1 (8.70) 


according to which œ; exp( jd) plays the role of a weighting factor. One output complex 
envelope H(t) corresponds to the transmission of symbol 0, and the other #,(t) corresponds 
to the transmission of symbol 1. The real parts of p(t) and #,(t) are then used in the 
decision-making process. The situation described here applies to binary FSK. In the case 
of binary PSK, only a single matched filter is needed, in which case the linear combiner 
produces a single output complex envelope. Here again, however, the real part of the 
combiner output is used in the decision-making process. 

In the linear combiner described herein, the “instantaneous” output signal-to-noise 
ratio (SNR) is the sum of the instantaneous SNRs on the individual diversity branches 
(channels). This optimum form of a linear combiner is therefore referred to as a maximal- 
ratio combiner; see Problem 8.17. 

Figure 8.24 shows the noise performance of coherent binary PSK, binary DPSK, and 
noncoherent binary FSK for L = 2, 4 independently fading channels. For the sake of 
comparison, we have also included in this figure the corresponding graphs for a fading 
channel with no diversity (i.e., L = 1). Figure 8.24 clearly illustrates the effectiveness of 
diversity as a means of mitigating the short-term effects of Rayleigh fading. 


8.8 TDMA and CDMA Wireless 
Communication Systems? 


In wireless communications, as with ordinary telephony, a user would like to talk and 
listen simultaneously. To cater to this natural desire, some form of duplexing is required. 
One way in which this requirement can be satisfied is to provide two frequency bands, 
one for the forward link from the base station to a mobile and the other for the reverse 
link from the mobile to the base station. As pointed out earlier, in North America the 
subband 869-894 MHz is used for the forward link, and the subband 824-849 MHz is 
used for the reverse link. This form of duplexing is called frequency division duplexing 
(FDD). Indeed, FDD is an integral part of the two widely used wireless communication 
systems summarized in Table 8.3. ` 

The first of these systems, namely, GSM, uses TDMA. From Section 8.2 we recall 
that in a TDMA system each subscriber is permitted to access the radio channel during a 
set of predetermined time slots, during which time that particular subscriber will have full 
use of the channel. Consequently, data are transmitted over the channel in bursts, as shown 
in the frame structure of Figure 8.25. The basic frame of GSM is composed of eight 
577 us slots. The 1-bit flag preceding each data burst of 57 bits is used to identify whether 
the data bits are digitized speech or some other information-bearing signal. The 3 tail bits, 
all logical zeros, are used in convolutional decoding of the channel-encoded data bits. 
(Convolutional codes are discussed in Chapter 10.) The 26-bit training sequence in the 
middle of the time slot is used for channel equalization. Finally, the guard time, occupying 
8.25 bits, is included at the end of each slot to prevent data bursts received at the base 
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TABLE 8.3 Summary of two widely used 


wireless communication systems 





Item GSM? IS-95t Comments 
Number of duplex 125 20 CDMA assumes 12.5 MHz in each direction; 
channels see the next line 
Channel bandwidth 200 1,250 
(kHz) 
Type of multiple access TDMA CDMA 
Access users per 8 20 to 35 A TDMA system is deterministic in that the 
channel number of access users per channel is defined 


by the number of available time slots. On 
the other hand, a CDMA system is 
interference-limited in that it has. a soft limit 
on the number of access users per channel, 

Modulation type GMSK BPSK/QPSK In CDMA, data are modulated as BPSK, but 
the spreading is QPSK 

Data rate (kb/s) 270.833 9.6 or 14.4 

Frame period (ms) 4.615 20 For CDMA, the frame period equals that of 
the speech codec (coder/decoder) 


5 a a aA a a a a i a ea E 
*GSM< stands for Global System for Mobile Communications; originally, it was introduced as an acronym for 


Groupe de travail Spéciale pour les services Mobiles. 
HIS stands for Interim Standard. 


station from mobiles from overlapping with each other; this is achieved by transmitting 
no signal at all during the guard time. With each slot consisting of 156.25 bits, of which 
40.25 bits are overhead (ignoring the 2 flag bits), the frame efficiency of GSM is 


_ 40.25 
AY” 156.25 


The second wireless communication system, IS-95, summarized in Table 8.3 uses 
CDMA. From Section 8.2 we recall that in CDMA, each subscriber is assigned a distinct 
spreading code (PN sequence), thereby permitting the subscriber full access to the channel 
all of the time. Consequently, in a CDMA system we have a new form of interference 
called multiple-access interference (MAI), which arises because of deviation of the spread- 
ing codes from perfect orthogonality. A related phenomenon that needs attention is the 
near-far problem, which occurs if the received signals from the mobile units do not have 
equal power at the base station. In such a situation, the strongest received signal from a 
mobile user captures the demodulation process at the base station to the detriment of the 
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FIGURE 8.25 Frame structure of the GSM wireless communication system. 
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other users. To overcome the near-far problem, it is customary to use power control at 
the base station, whereby the base station maintains control over the power level of the 
transmitted signal from every mobile being served by that base station. The use of power 
control is particularly important in CDMA systems for another reason. A goal of multiple- 
access systems is to maximize system capacity, which is defined as the largest possible 
number of users that can be reliably served by the system, given prescribed resources. 
Clearly, system capacity is compromised if each mobile is free to raise its transmitted power 
level regardless of other users, since that increase in transmitted power will, in turn, raise 
the level of multiple-access interference in the system. To maximize system capacity, it is 
therefore essential that each mobile’s transmitter be under the control of the serving base 
station so that the signal-to-interference ratio is maintained at the minimum acceptable 
level needed for reliable service. 


# RAKE RECEIVER 


A discussion of wireless communications using CDMA would be incomplete without a 
description of the RAKE receiver.° The RAKE receiver was originally developed in the 
1950s as a “diversity” receiver designed expressly to equalize the effect of multipath. First, 
and foremost, it is recognized that useful information about the transmitted signal is con- 
tained in the multipath component of the received signal. Thus, taking the viewpoint that 
multipath may be approximated as a linear combination of differently delayed echoes, the 
RAKE receiver seeks to combat the effect of multipath by using a correlation method to 
detect the echo signals individually and then adding them algebraically. In this way, in- 
tersymbol interference due to multipath is dealt with by reinserting different delays into 
the detected echoes so that they perform a constructive rather than destructive role. 
Figure 8.26 shows the basic idea behind the RAKE receiver. The receiver consists of 
a number of correlators connected in parallel and operating in a synchronous fashion. 
Each correlator has two inputs: (1) a delayed version of the received signal and (2) a replica 
of the pseudo-noise (PN) sequence used as the spreading code to generate the spread- 
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FIGURE 8.26 Block diagram of the RAKE receiver. 
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spectrum modulated signal at the transmitter. In effect, the PN sequence acts as a “refer. 
ence signal.” Let the nominal bandwidth of the PN sequence be denoted as W = 1/7 
where T, is the chip duration. From the discussion of spread-spectrum modulation pre. 
sented in Chapter 7, we recall that the autocorrelation function of a PN sequence has a 
single peak of width 1/W, and it disappears toward zero elsewhere inside one periad of 
the PN sequence (i.e., one symbol period). Thus we need only make the bandwidth W of 
the PN sequence sufficiently large to “identify” the significant echoes in the received signa], 
To be sure that the correlator outputs all add constructively, two other operations are 
performed in the receiver by the functional blocks labeled “phase and gain adjustors”. 


1. An appropriate delay is introduced into each correlator output so that the phase 
angles of the correlator outputs are in agreement with each other. 

2. The correlator outputs are weighted so that the correlators responding to strong 
paths in the multipath environment have their contributions accentuated, while the 
correlators not synchronizing with any significant path are correspondingly 
suppressed. 


The weighting coefficients, œp, are computed in accordance with the maximal ratio com. 
bining principle:'® 


The signal-to-noise ratio of a weighted sum, where each element of the sum consists 
of a signal plus additive noise of fixed power, is maximized when the amplitude 
weighting is performed in proportion to the pertinent signal strength. 


The linear combiner output is 


M 
y(t) = 2 azz, (t) (8.71) 


where z(t) is the phase-compensated output of the kth correlator, and M is the number 
of correlators in’the receiver. Provided we use enough correlators in the receiver to span 
a region of delays sufficiently wide to encompass all the significant echoes that are likely 
to occur in the multipath environment, the output y(t) behaves essentially as though there 
was a single propagation path between the transmitter and receiver rather than a series of 
multiple paths spread in time. 

To simplify the presentation, the receiver of Figure 8.26 assumes the use of binary 
phase-shift keying in performing spread-spectrum modulation at the transmitter. Thus the 
final operation performed in Figure 8.26 is that of integrating the linear combiner output 
y(t) over the bit interval T, and then determining whether binary symbol 1 or 0 was 
transmitted in that bit interval. 

The RAKE receiver derives its name from the fact that the bank of parallel correlators 
has an appearance similar to the fingers of a rake. Because spread spectrum modulation 
is basic to the operation of CDMA wireless communications, it is natural for the RAKE 
receiver to be central to the design of the receiver used in this type of multiuser radio 
communication.’ 


8.9 Source Coding of Speech for 
Wireless Communications 


For the efficient use of channel bandwidth, digital wireless communication systems, be 
they of the TOMA or CDMA type, rely on the use of speech coding to remove almost all 
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of the natural redundancy in speech, while maintaining a high-quality speech on decoding. 

The common approach is to use source coding, which, in one form or another, exploits 
the linear predictive coding (LPC) of speech. 

In this section, we describe two different techniques for speech coding: multi-pulse 

excited LPC and code-excited LPC, versions of which are used in GSM and IS-95, respec- 

` tively. Our treatment of both of these speech coding techniques is in conceptual terms.’ 


= MULTI-PULSE ExcirED LPC 


This form of speech coding exploits the principle of analysis by synthesis, which means 
that the encoder includes a replica of the decoder in its design. Specifically, the encoder 
consists of three main parts as indicated in Figure 8.27a: 


1. Synthesis filter for the predictive modeling of speech. It may consist of an all-pole 
filter (i.e., a filter whose transfer function has poles only), which is designed to model 
the short-term spectral envelope of speech; the term short-term refers to the fact that 
the filter parameters are computed on the basis of predicting the present sample of the 
speech signal using eight to sixteen previous samples. The synthesis filter may also 
include a long-term predictor for modeling the fine structure of the speech spectrum; 
in such a case, the long-term predictor is connected in cascade with the short-term 
predictor. In any event, the function of the synthesis filter is to produce a synthetic 
version of the original speech that is of high quality. 

Excitation generator for producing the excitation applied to the synthesis filter. The 

excitation consists of a definite number of pulses every 5 to 15 ms. The amplitudes 

and positions of the individual pulses are adjustable. 

3. Error minimization for optimizing the perceptually weighted error between the orig- 
inal speech and synthesized speech. The aim of this minimization is to optimize the 
amplitudes and positions of the pulses used in the excitation. Typically, a mean- 
square error criterion is used for the minimization. 


P 


Thus, as shown in Figure 8.27a, the three parts of the encoder form a closed-loop 
optimization procedure, which permits the encoder to operate at a bit rate below 16 
kb/s, while maintaining high-quality speech. 

The encoding procedure itself has two main steps: 


* The free parameters of the synthesis filter are computed using the actual speech 
samples as input. This computation is performed outside the optimization loop over 
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FicurE 8.27 Muiti-pulse excited linear predictive codec. (a) Encoder. (b) Decoder whose input 
(the received signal) consists of quantized filter parameters and quantized excitation as produced 
by the encoder. 
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a period of 10 to 30 ms, during which the speech signal is treated as pseudo. 
stationary. 

e The optimum excitation for the synthesis filter is computed by minimizing the per. 
ceptually weighted error with the loop closed as in Figure 8.274. 


Thus the speech samples are divided into frames (10 to 30 ms long) for computing the 
filter parameters, and each frame is divided further into subframes (5 to 15 ms) for opti- 
mizing the excitation. The quantized filter parameters and quantized excitation constitute 
the transmitted signal. 

Note that by first permitting the filter parameters to vary from one frame to the next, 
and then permitting the excitation to vary from one subframe to the next, the encoder is 
enabled to track the nonstationary behavior of speech, albeit on a batch-by-batch basis, 

The decoder, located in the receiver, consists simply of two parts: excitation generator 
and synthesis filter, as shown in Figure 8.27b. These two parts are identical to the corre- 
sponding ones in the encoder. The function of the decoder is to use the received signal to 
produce a synthetic version of the original speech signal. This is achieved by passing the 
decoded excitation through the synthesis filter whose parameters are set equal to those in 
the encoder. 

To reduce the computational complexity of the codec (i.e., contraction of coder/ 
decoder), the intervals between the individual pulses in the excitation are constrained to 
assume a common value. The resulting analysis-by-synthesis codec is said to have a regular- 
pulse excitation. ' 


CoDE-EXCITED LPC 


Figure 8.28 shows the block diagram of the code-excited LPC, commonly referred to as 
CELP. The distinguishing feature of CELP is the use of a predetermined codebook of 
stochastic (zero-mean white Gaussian) vectors as the source of excitation for the synthesis 
filter. The synthesis filter itself consists of two all-pole filters connected in cascade, one of 
which performs short-term prediction and the other performs long-term prediction. 

As with the multi-pulse excited LPC, the free parameters of the synthesis filter are 
computed first, using the actual speech samples as input. Next, the choice of a particular 
vector (code) stored in the excitation codebook and the gain factor G in Figure 8.28 is 
optimized by minimizing the average power of the perceptually weighted error between 
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FIGURE 8.28 Encoder of the code-excited linear predictive codec (CELP): the transmitted sig- 
nal consists of the address of the code selected from the codebook, quantized G, and quantized 
filter parameters. 
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the original speech and synthesized speech (i.e., output of the synthesis filter). The address 
of the stochastic vector selected from the codebook and the corresponding quantized gain 
factor, together with the quantized filter parameters, constitute the transmitted signal. 

An identical copy of the codebook is made available to the decoder, and likewise for 
the synthesis filter. Hence, given the received signal, the decoder is enabled to parameterize 
its own synthesis filter and determine the appropriate excitation for the synthesis filter, 
thereby producing a synthetic version of the original speech signal. 

CELP is capable of producing good-quality speech at bit rates below 8 kb/s. How- 
ever, its computational complexity is intensive because of the exhaustive search of the 
excitation codebook. In particular, the weighted synthesized speech in the encoder has to 
be computed for all the entries in the codebook and then compared with the weighted 
original speech. Nevertheless, real-time implementation of CELP codecs has been made 
possible by virtue of advances in digital signal processing and VLSI technology. 


8.10 Adaptive Antenna Arrays for 
Wireless Communications” 


The goal of wireless communications is to allow as many users as possible to communicate 
reliably without regard to location and mobility. From the discussion presented in Sections 
8.5 and 8.6, we find that this goal is seriously impeded by three major channel impairments: 


1. Multipath can cause severe fading due to phase cancellation between different prop- 
agation paths. Fading leads to a reduction in available signal power and therefore a 
degraded noise performance. 


2. Delay spread results from differences in propagation delays among the multiple prop- 
agation paths, When the delay spread exceeds about 10 percent of the symbol du- 
ration, the intersymbol interference experienced by the received signal reaches a sig- 
nificant level, thereby causing a reduction in the attainable data rate. 

3. Co-channel interference arises in cellular systems where the available frequency chan- 
nels are divided into different sets, with each set being assigned to a specific cell and 
with several cells in the system using the same set of frequencies. Co-channel inter- 
ference limits the system capacity (i.e., the largest possible number of users that can 
be reliably served by the system). 


Typically, cellular systems use 120° sectorization at each base station, and only 
one user accesses a sector of a base station at a given frequency. We may combat the 
effects of multipath fading and co-channel interference at the base station by using three 
identical but separate antenna arrays, one for each section of the base station. The 
compensation of delay spread is considered later in the section. Figure 8.29 shows the 
block diagram of an array signal processor, where it is assumed that there are N users 
whose signals are received at a particular sector of the base station, and the array for that 
sector consists of M identical antenna elements. A particular user is treated as the one of 
interest, and the remaining N-1 users give rise to co-channel interference. In addition to 
the co-channel interference, each component of the array signal processor’s input is cor- 
rupted by additive white Gaussian noise (AWGN). The analysis presented herein is for 
baseband signals, which, in general, are complex valued. This, in turn, means that both 
the channel and array signal processor require complex characterizations of their own. 
The structure depicted in Figure 8.29 is drawn for one output pertaining to the user of 
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FIGURE 8.29 Block diagram of array signal processor that involves M antenna elements, and 
that is being driven by a multipath channel. 


interest. The array signal processor is duplicated for users at other frequencies at the base 
station. 

The multipath channel is characterized by the channel matrix, which is denoted by 
C. The matrix C.has dimensions M-by-N and may therefore be expanded into N column 
vectors, as shown by 


C= [c1, C23». .3 cn] (8.72) 


where each column vector is of dimension M. . 
Given the configuration described in Figure 8.29, the goal is to design a linear array 
signal processor for the receiver, which satisfies two requirements: 


1. The co-channel interference produced by the N-1 interfering users is cancelled. 
2. The output signal-to-noise ratio (SNR) for the user of interest is maximized. 


Hereafter, these two requirements are referred to as design requirements 1 and 2. 

To proceed with this design task, it is assumed that the multipath channel is described 
by flat Rayleigh fading. Then, in light of the material presented in Section 8.7, we find that 
the use of diversity permits the treatment of the column vectors c4, C2, ..., Cn as linearly 
independent, which is justified provided that the spacing between antenna elements of the 
array is large enough (e.g., seven times the wavelength) for independent fading. To simplify 
the presentation, we suppose that user 1 is the user of interest and the remaining N ~ 1 
users are responsible for co-channel interference, as indicated in Figure 8.29. The key 
design issue is how to find the weight vector denoted by w, which characterizes the array 
signal processor. To that end, we may proceed as follows: 


1. We choose the M-dimensional weight vector w to be orthogonal to the vectors 
2, .. - , x, Which are associated with the interfering users. This choice fulfills design 
requirement 1 (i.e., cancellation of co-channel interference). 

2. To satisfy design requirement 2 (i.e., maximization of the SNR), we will briefly di- 
gress from the issue at hand to introduce the notion of a subspace. Given a vector 
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space, or just space, formed by a set of linearly independent vectors, a subspace of 
the space is a subset that satisfies two conditions: !* 
(i) If we add any two vectors z, and z; in the subspace, their sum 2, and z; is still 
in the subspace. 
(ii) If we multiply any vector z in the subspace by any scalar a, the multiple az is 
still in the subspace. 


Returning to the issue of how to maximize the output SNR for user 1, we first 
construct a subspace denoted by W, whose dimension is equal to the difference be- 
tween the number of antenna elements and the number of interfering users, that is, 
M- (N - 1) = M ~ N + 1. Next, we project the complex conjugate of the channel 
vector c4 (pertaining to user 1) onto the subspace W. The projection so computed 
defines the weight vector w. 


b EXAMPLE 8.3 


To illustrate the two-step subspace method for determining the weight vector w, consider the 
simple example of a system involving two users characterized by the channel vectors c and 
cz, and an antenna array consisting of three elements; that is, N = 2 and M = 3. Then, for 
this example, the subspace W is two-dimensional, as shown by 


M-N+1=3-2+1=2 
With user 1 viewed as the user of interest and user 2 viewed as the interferer, we may construct 
the signal-space diagram shown in Figure 8.30. The subspace W, shown shaded in this figure, 
is orthogonal to channel vector cz. The weight vector w of the array signal processor is de- 
termined by the projection of the complex-conjugated channel vector of user 1, that is, cf, 
onto the subspace W, as depicted in Figure 8.30. k 


The important conclusion drawn from this discussion is that a linear receiver using 
optimum combining with M antenna elements and involving N — 1 interfering users has 
the same performance as a linear receiver with M — N + 1 antenna elements without 
interference, independent of the multipath environment. For this equivalence to be realized, 
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FIGURE 8.30 Signal-space diagram for Example 8.3, involving a user of interest, a single inter- 
ferer, and an antenna array of 3 elements. The subspace W, shown shaded, is two-dimensional in 
this example. 
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we of course require that M > N — 1. Provided that this condition is satisfied, the receiver 
cancels the co-channel interference with a diversity improvement equal to M — N + 4 
which represents an N-fold increase in system capacity. i 

The design of an array signal processor in accordance with the two-step subspace 
procedure described herein is of the zero-forcing kind. We say so because, given M antenna 
elements, the array has enough degrees of freedom to force the output due to the N ~ 1 
interfering users represented by the linearly independent channel vectors c2,..., cy to 
zero so long as M is greater than N — 1. Note also that this procedure includes N = 1 
(i.e., a single user with no interfering users) as a special case. In this case, the channel 
matrix consists of vector ¢,, which lies in the subspace W, and the zero-forcing solution 
w equals cj. 

The analysis presented thus far has been entirely of a spatial kind, which ignores the 
effect of delay spread. What if the delay spread is significant compared to the symbol 
duration and cannot therefore be ignored? Recognizing that delay spread is responsible 
for intersymbol interference, we may, in light of the material presented in Chapter 4 on 
the equalization of a telephone channel, incorporate a linear equalizer in each antenna 
branch of the array to compensate for delay spread. The resulting array signal processor 

“takes the form shown in Figure 8.31, which combines temporal and spatial processing, 
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FIGURE 8.31 Baseband space-time processor. The blocks labeled z~? are unit-delay elements 
with each delay being equal to the symbol period. The filter coefficients are complex valued. The 
FIR filters are all assumed to be of length L. 
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Spatial processing is provided by the antenna array, and the temporal processing is pro- 
vided by a bank of finite-duration impulse response (FIR) filters. For obvious reasons, this 
structure is called a space-time processor. 


ADAPTIVE ANTENNA ARRAY 


The subspace design procedure for the array signal processor in Figure 8.29 assumes that 
the channel impairments are stationary, and that we have knowledge of the channel matrix 
C. In reality, however, multipath fading, delay spread, and co-channel interference are all 
nonstationary in their own individual ways. Also, the channel characterization may be 
unknown. To deal with these practical issues, we need to make the receiving array signal 
processor in Figure 8.29 adaptive, Bearing in mind the scope of this book, we confine the 
discussion to adaptive spatial processing, assuming that the delay spread is negligible. We 
further assume that the multipath fading phenomenon is slow enough to justify the least- 
mean-square (LMS) algorithm to perform the adaptation. 

Figure 8.32 shows the structure of an adaptive antenna array, where the output of 
each antenna element is multiplied by an adjustable (controllable) weight, and then the 
weighted elemental outputs of the array are summed to produce the array output signal. 
The adaptive antenna array does not require knowledge of the direction of arrival of the 
desired signal originating from a user of interest as long as the system is supplied with a 
reference signal, which is correlated with the desired signal. The output signal of the array 
is subtracted from the reference signal to generate an error signal, which is used to apply 
the appropriate adjustments to the elemental weights of the array. In this way, a feedback 
system to control the elemental weights is built into the operation of the antenna array, 
thereby making it adaptive to changes in the environment. Note that the block diagram 
of Figure 8.32 is drawn for baseband processing, hence the complex conjugation of the 
elemental weights. In a practical system, a quadrature hybrid is used for each antenna 
element of the array to split the complex-valued received signal at each element into two 
components: one real and the other imaginary. The use of a hybrid has been omitted in 
Figure 8.32 to simplify the diagram. 
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FIGURE 8.32 Block diagram of adaptive antenna array. 
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To optimize the performance of the adaptive antenna array, it is customary to use 

the mean-square error 

J = Eilein |") (8.73) 
as the cost function to be minimized. The e[n] is the error signal at time £ = nT, where T 
is the symbol period and z is an integer serving as discrete time. Minimization of the cost 
function J suppresses the interfering signals and enhances the desired signal in the array 
output. However, the LMS algorithm minimizes the instantaneous value of the cost func- 
tion J and, through successive iterations, it strives to reach the minimum mean-square 
error (MMSE) (i.e., optimum solution for the elemental weights). In light of the discussion 
presented in Chapter 4 on temporal equalizers, which carries over to the spatial domain, 
we may say that an adaptive antenna array based on the minimum mean-square Eitor 
criterion is highly likely to provide a better solution than one based on the zero-forcing 
criterion embodied in the two-step subspace method. 

Let x[n] denote the output of the kth element in the array at discrete time 1, and let 
wain] denote the corresponding value of the weight connected to this element. The output 
signal of the array (consisting of M antenna elements) is therefore 

M 


yin] = È wln (8.74) 


k=1 


where w*[n]x,[7] is the inner product of the complex-valued quantities wln] and x[n]. 
Denoting the reference signal as din], we may evaluate the error signal as 


e[n) = din] — yin] (8.75) 
Hence, the adjustment applied to the kth elemental weight is , 
Awla] = ue*[njedn], k= 1, 2,3 M (8.76) 

where p is the step-size parameter, and the updated value of this weight is 
win + 1] = wln] + Awl, k=1,2,...,M (8.77) 


Equations (8.74)-(8.77), in that order, constitute the complex LMS algorithm, which in- 
cludes the LMS algorithm for real signals (studied in Chapters 3 and 4) as a special case. 
The algorithm is initiated by setting w,{0] = 0 for all k. The derivation of the complex 
LMS algorithm is posed as Problem 8.19. 

The advantages of an adaptive antenna array using the complex LMS algorithm are 
three-fold: 


> Simplicity of implementation. 

> Linear growth in complexity with the number of antenna elements. 

» Robust performance with respect to disturbances. 

However, the system suffers from the following drawbacks: 

» Slow rate of convergence, which is typically ten times the number of weights. This. 
limits the use of the complex LMS algorithm to a slow-fading environment, for which 
the Doppler spread is small compared to the reciprocal of the duration of the obser- 
vation interval. 

> Sensitivity of the convergence behavior to variations in the reference signal and co- 
channel interference powers. 

These limitations of the complex LMS algorithm can be overcome by using an al- 


gorithm known as direct matrix inversion (DMI), which follows directly from the Wienet 
filter discussed in Chapter 4; see Problem 8.21. Unlike the LMS algorithm, the DMI al 
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gorithm operates in the batch mode in that the computation of the elemental weights is 
based on a batch of K snapshots. The batch size K is chosen as a compromise between 
two conflicting requirements: 


> The size K should be small enough for the batch of snapshots used in the computation 
to be justifiably treated as pseudo-stationary. 


> The size K should be large enough for the computed values of the elemental weights 
to approach the MMSE solution. 


The DMI algorithm is the optimum combining technique for array antennas currently 
deployed in many base stations today. The DMI algorithm may be reformulated for re- 
cursive computation,’ if so desired. 

When the teletraffic is high, the base stations are ordinarily configured as microcells, 
which are small cells such as an office floor or a station deployed along a highway with 
directional antennas. In such a configuration, there are many inexpensive base stations in 
close proximity to each other. The use of adaptive antenna arrays provides the means for 
an alternative configuration where there are fewer (but more expensive) base stations and 
further apart from each other than in the corresponding microcellular system. 


i 8. 11 Summary and Discussion 


In this chapter, we discussed two important types of multiuser communications: satellite 
communications and wireless communications. Satellite communication systems offer 
global coverage, whereas wireless communication systems offer mobility. The global cov- 
erage and mobility offered by these two communication systems have profoundly trans- 
formed the way we communicate, both locally and globally. 

Although satellite communication and wireless communication systems function in 
entirely different ways, both rely on radio propagation to link the receiver to the trans- 
mitter. In satellite communications, we have an uplink from an earth terminal to the 
satellite transponder and a downlink from the satellite to another earth terminal. The 
satellite operates like a repeater in the sky. Moreover, with the satellite positioned in a 
geostationary orbit, the uplink and downlink operate as line-of-sight paths of fixed lengths. 
Accordingly, the satellite communication channel, encompassing both of these links, is 
closely modeled as an additive white Gaussian noise (AWGN) channel. 

The wireless communication system also has two links of its own: an uplink, or 
reverse link, for the mobile-to-base station transmission, and a downlink, or forward link, 
for the base station-to-mobile transmission. The base station is fixed, being located at the 
center or on the edge of a coverage region; it consists of radio channels, and transmitter, 
and receiver antennas mounted on a tower. Three major sources of degradation in wireless 
communications, discussed in the chapter, are co-channel interference, fading, and delay 
spread; the latter two are byproducts of multipath. A common characteristic of these 
channel impairments is that they are all signal-dependent phenomena. Unlike the ubiqui- 
tous channel noise, the degrading effects of interference and multipath cannot therefore 
be combatted by simply increasing the transmitted signal power. Rather, both interference 
and multipath require the use of specialized techniques, tailor-made to their particular 
physical characteristics. These specialized techniques include diversity, adaptive array an- 
tennas, and the RAKE receiver. 

We close the discussion with remarks contrasting wireless communications to wired 
communications. From Chapter 3 we recall that a major source of concern in wired com- 
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munication systems is noise; these systems have sufficient channel bandwidth to permit 
the use of pulse-code modulation (PCM) as the standard method for converting speech 
into a 64 kb/s stream, which provides the basic data for an almost noise-free performance, 
In wireless communications, on the other hand, channel bandwidth is a precious resource, 
the conservation of which necessitates the use of spectrally efficient speech coding tech. 
niques to produce tol!-quality digitized speech at rates that are a small fraction of the PCM 
rate, Unfortunately, the waveform coders exemplified by adaptive differential pulse-code 
modulation, discussed in Chapter 3, do not satisfy this stringent requirement. The preferred 
approach is to use the spectrally efficient source-coding techniques: multi-pulse excited 
linear predictive coding (LPC) or its regular-pulse excited variant, and code-excited LPC 
(CELP); these source coding techniques produce bit rates below 16 kb/s by removing 
almost all of the natural redundancy in speech, while maintaining high-quality speech, 
albeit of a synthetic kind. To provide protection against noise, channel coding is used 
whereby redundant bits are inserted into the transmitted data stream in a controlled man- 
ner. The use of channel coding also helps in other ways: It extends the range of low-power 
handsets as well as battery life. Channel coding is discussed in Chapter 10. 


i NOTES AND REFERENCES 


1. For detailed treatment of satellite communications and related issues, see the following 
books: Sklar (1988), Pratt and Bostian (1986), Wu (1984), Bhargava et al. (1981), and 
Spilker, Jr. (1977). The first, third, fourth, and fifth books emphasize the use of satellites 
for digital communications. The book by Pratt and Bostian presents a broad treatment of 
satellite communications, emphasizing such diverse topics as radio-wave propagation, an- 
tennas, orbital mechanics, signal processing, and radio electronics. > 


2. Link budget analysis is discussed in the books by Sklar (1988) and Anderson (1999); for 
satellite communications, it is discussed in Bhargava et al. (1981). 


3. For the fundamentals of antennas, see the book by Kraus (1950) and Chapter 11 of the 
book by Jordan and Balmain (1973). 


4. The free-space equation (Equation 8.14) is named in honor of Friis (1946), For the origin 
of the Friis formula of Equation (8.30), see Friis (1944). 


5. For an original treatment of cellular radio, see the paper by MacDonald (1979). 


6. For a comprehensive treatment of the mobile radio propagation channel, see the book by 
Parsons (1992). This book presents the fundamentals of VHF and UHF propagation, prop- 
agation over irregular terrain and in built-up areas, and a statistical characterization of the 
mobile radio channel. The statistical characterization of a mobile radio channel is also 
discussed in Proakis (1995). This book provides a readable account of the effect of fading 
on the error performance of Rayleigh fading channels and a good discussion of diversity 
techniques. For a full treatment of the subject, see Chapters 9-11 by Stein in the book 
edited by Schwartz, Bennett, and Stein (1966). 


7. The chi-square distribution is a special case of the gamma distribution. The probability 
density function of a gamma-distributed random variable X has two parameters: a > 0 
and A > 0; it is defined by 


a—1 -àx 
fo = PE, 0<x< a 


11. 


12. 


13. 


14. 


15. 


16. 
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where I'(a) is the gamma function, which is itself defined by 


Ta) = | ze “dz, a>O 


The gamma function has the following properties: 
(1/2) = Vr 
T(a + 1) = aT (a), ao > 0 


By letting A = 1/2 and a = k/2, where k is a positive integer, we get the chi-square distri- 
bution with 2k degrees of freedom, as shown by 


lFO22 gla 


fel) = Sar ED)? O<x<o@ 


. For a survey article on the evolution of wireless communications, see Oliphant (1999). For 


books on the fundamentals of wireless communication systems, see Steele and Hanzo 
(1999), Stüber (1996) and Rappaport (1996). 

For a detailed description of GSM, see Chapter 8 of the book by Steele and Hanzo 
(1999). For a detailed description of the IS-95 system, see the handbook by Lee and Miller 
(1998). 


. The classic paper on the RAKE receiver is due to Price and Green (1958). 
10. 


For the original paper on how to maximize the signal-to-noise ratio realizable from the 
sum of several noisy signals, see the classic paper by Brennan (1955). 


The application of the RAKE receiver in CDMA wireless communication systems is dis- 
cussed in detail in the book by Viterbi (1995). 


The idea of multi-pulse excitation for speech coding is due to Atal and Remde (1982). 
Code-excited linear prediction (CELP) of speech was first introduced by Atal and Schroeder 
(1984). For a detailed mathematical discussion of multi-pulse excited, regular-pulse ex- 
cited, and code-excited types of speech coding, particularly as they relate to wireless com- 
munications, see Chapter 3 in the book edited by Steele and Hanzo (1999). 


In the wireless communications literature, adaptive antenna arrays are often referred to as 
smart antennas. For an overview of the various issues involved in the use of adaptive 
antenna arrays for wireless communications, see the article by Winters (1998) and the 
course notes by Winters (1999). The two-step subspace procedure for designing the array 
signal processor in Figure 8.29 is based on material presented in Winters (1999). The book 
by Rappaport (1999) presents a collection of papers on adaptive antenna arrays, which 
are grouped into algorithms, architectures, hardware applications, channel models, and 
performance evaluation. 


The idea of subspace is rooted in matrix algebra. For a discussion of this idea, see Strang 
(1980) and Stewart (1973). For a discussion of subspace decomposition in the context of 
statistical signal processing, see Scharf (1991). 


For tutorial discussions of space-time processing for wireless communications, see the ar- 
ticles by Paulraj and Ng (1998), Paulraj and Papadias (1997), and Kohno (1998). 


Recursive implementation of the DMI algorithm leads to a new algorithm commonly re- 
ferred to as the recursive least squares (RLS) algorithm; for a derivation of the RLS algo- 
rithm and its variants, see Haykin (1996). 
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i PROBLEMS 


Free-Space Propagation 


8.1 


8.2 
8.3 


8.4 


8.5 


8.6 


A radio link uses a pair of 2m dish antennas with an efficiency of 60 percent each, as 
transmitting and receiving antennas. Other specifications of the link are: 


Transmitted power = 1 dBw 
Carrier frequency = 4 GHz 
Distance of the receiver 

from the transmitter = 150 m 


Calculate (a) the free-space loss, (b) the power gain of each antenna, and (c) the received 
power in dBW. 


Repeat Problem 8.1 for a carrier frequency of 12 GHz. 


Equation (8.14) is one formulation of the Friis free-space equation. Show that this equa- 
tion can also be formulated in the following equivalent forms: 








PAA, 
(a) P, = Peo 

_ BAG, 
(b) P, = Gop 


where P, is the transmitted power, A, is the effective area of the transmitting antenna, A 
is the carrier wavelength, d is the distance of the receiver from the transmitter, G, is the 
power gain of the receiving antenna, A, is the effective area of the receiving antenna, and 
P, is the received power. 

Discuss the situations that favor the use of one of these equations over the other. 


From the mathematical definition of the free-space loss 


4nd\” 
Lecce space 7 (=) 


we see that it is dependent on the carrier wavelength A or frequency f. How can this 
dependence on wavelength or frequency be justified in physical terms? 


In a satellite communication system, the carrier frequency used on the uplink is always 
higher than the carrier frequency used on the downlink. Justify the rationale for this 
choice. 


A continuous-wave (CW) beacon transmitter is located on a satellite in geostationary 
orbit. The beacon’s 12 GHz output is monitored by an earth station positioned 40,000 
km from the satellite. The satellite transmitting antenna is a 1m dish with an aperture 
efficiency of 70 percent, and the earth station receiving antenna is a 10m dish with an 
aperture efficiency of 55 percent. Calculate the received power, given that the beacon’s 
output power is 100 mW. 


Noise Figure 


8.7 


Consider a 75-Q resistor maintained at “room temperature” of 290K. Assuming a band- 
width of 1 MHz, calculate the following: 


(a) The root-mean-square (RMS) value of the voltage appearing across the terminals of 
this resistor due to thermal noise. 


(b) The maximum available noise power delivered to a matched load. 
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8.8 In this problem, we revisit Example 8.1 based on the receiver configuration of Figure 
8.10. Suppose that a lossy waveguide is inserted between the receiving antenna and the 
low-noise amplifier. The waveguide loss is 1 dB, and its physical temperature is 290K. 
Recalculate the effective noise temperature of the receiver. 


8.9 Consider the receiver of Figure P8.9, which consists of a lossy waveguide, low-noise RF 
amplifier, frequency down-converter (mixer), and IF amplifier. The figure includes the 
noise figures and power gains of these four components. The antenna temperature is 50K. 
(a) Calculate the equivalent noise temperature for each of the four components in Figure 

P8.9, assuming a room temperature T = 290K. 
(b) Calculate the effective noise temperature of the whole receiver. 

















n Frequency Intermediate 
ented —> down- frequency Output 
Fa? P converter amplifier 
Gate F=17 F=3 F=5 
G=10 G=5 G = 5,000 
FIGURE P8.9 


Budget Link Calculations 


8.10 In this problem we address the uplink power budget of the digital satellite communication 
system considered in Example 8.2. The parameters of the link are as follows: 


Carrier frequency = 14 GHz 
Power density at the TWT 

amplifier in saturation = —81 dBW/m* 
Satellite figure of merit, G/T = 1.9 dB/K 
Distance of the satellite from the 

transmitting earth terminal = 40,000 km 


(a) Assuming no power backoff of the TWT, calculate the C/N, ratio at the satellite. 
(b) Given that the data rate in the uplink is the same as that calculated for the downlink 
in Example 8.2, calculate the probability of symbol error incurred in the uplink al- 
lowing for a link margin of 6 dB. Compare your result with that in Example 8.2. 
8.11 The downlink C/Np ratio in a direct broadcast satellite (DBS) system is estimated to be 
85 dB-Hz. The specifications of the link are: 


Satellite EIRP = 57 dBW 
Downlink carrier frequency = 12.5 GHz 
Data rate = 10 Mb/s 
Required E,/Np at the receiving earth terminal = 10 dB 


Distance of the satellite from the receiving earth terminal = 41,000 km 


Calculate the minimum diameter of the dish antenna needed to provide a satisfactory TV 
reception, assuming that the dish has an efficiency of 55 percent and it is located alongside 
the home where the temperature is 310K. For this calculation, assume that the operation 
of the DBS system is essentially downlink-limited. 
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Wireless Communications 


8.12 


8.13 


8.14 


8.15 


8.16 


Both wireless communications and satellite communications rely on radio propagation 
for their operations. Summarize (a) the similarities of these two multiuser communication 
systems, and (b) the major differences that distinguish them from each other. 
In wireless communication systems, the carrier frequency on the uplink (reverse link) is 
smaller than the carrier frequency on the downlink (forward link). Justify the rationale 
for this choice. 
Figure P8.14 depicts the direct (line-of-sight) and indirect (reflected) paths of a radio link 
operating over a plane earth. The heights of the transmitting antenna at the base station 
and the receiving antenna of a mobile unit are h, and hm, respectively. Assume the 
following: 
> The reflection coefficient of the ground is —1. 
> The distance d between the two antennas is large enough to make the phase difference ¢ 
between the reflected and direct paths small compared to 1 radian, so that we may set 
sind = ¢. 
Hence, show that the received power P, is given by the approximation 
2 
bihm 
a 


where P, is the transmitted power, and G, and G, are the power gains of the transmitting 
base and mobile antennas, respectively. Compare this result with the Friis free-space 
equation. ' 





P, = PCG 











FIGURE P8.14 


The żwo-path model defined by the impulse response 
h(t) = a, (t — %) + a, exp(—j0) S(t — 7) 


is frequently used in the analytic treatment of wireless communication systems. The model 

parameters are the delay times 7, and 7,, the uniformly distributed phase 6, and the real 

coefficients a, and az. 

(a) Determine (i) the transfer function of the model, and (ii) its power-delay profile. 

(b) Show that the model exhibits frequency-selective fading due to variations in the co- 
efficients a, and a. 

In the RAKE receiver illustrated in Figure 8.26, each correlator is synchronized by in- 

serting the right delay into the received signal. 

(a) Show that, in theory, the same result is obtained by inserting the right delay into the 
reference signal {i.e., pseudo-noise sequence). 

(b) In practice, the preferred method is to use the procedure described in Figure 8.26. 
What reason can you suggest for this preference? 
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8.17 In this problem we study the maximal-ratio combining diversity scheme. To proceed, 
consider a set of noisy signals {x,(t)}%,, where x;(t) is defined by 


xt) = s(t) + a(t), j7=1,2,...,N 
Assume the following: 
» The signal components s,(t) are locally coherent, that is, 
s(t) = zm(t), jf=1,2,...,N 
where the z; are positive real numbers, and m(t) denotes a message signal with unit 


power. 
> The noise components 7;(t) have zero mean, and they are statistically independent, that is, 


of fork=j 
0 otherwise 


Elnj(t).(t)] = { 
The output of the linear combiner is defined by 
N 
x(t) = 5 a;x;(t) 
= : 


where the parameters a; are to be determined. 
(a) Show that the output signal-to-noise ratio is 


N 2 
2 a) 
j=1 


(SNR)o = — 
> ao} 
j=1 
(b) Set 
uj = a0; 
Zi 
v; = 
J o; 


and reformulate the expression for (SNR)o. Hence, applying the Schwarz inequality 
to this reformulation, show that 


N 
(i) (SNR)o = 2> (SNR); 
= 
where (SNR); = 2/07. 
(ii) The optimum values of the combiner’s coefficients are defined by 
Zi 


2 
oj 


a 
in which case the Schwarz inequality is satisfied with the equality sign. 


The Schwarz inequality is discussed in Section 5.2. 


Adaptive Antenna Arrays 


8.18 Consider the array signal processor of Figure 8.29 where there are only two users (N = 
2) and the array consists of two elements (M = 2). Construct the subspace W for this 
problem. Hence, using a signal-space diagram, illustrate the computation of the weight 


characterizing the array signal processor. 
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8.19 


8.20 


8.21 


In this problem we derive the complex LMS algorithm. Referring to Figure 8.32 ang 
starting with the instantaneous cost function 


J =$ jer]? 


where e[n] is the error signal and M is the number of antenna elements, do the following, 

(a) Determine the derivative of the cost function J with respect to the kth elemental weight 
wla]. 

(b) Using the instantaneous derivative dJ/dw,|7], denoted by VJ [k], determine the ad- 
justment Aw,[#] made to the kth elemental weight in accordance with the rule 


Aw,[n] = —uÎJik] 


(c) Verify the composition of the complex LMS algorithm described in Equations (8.75) 
to (8.77). 
Note that w,[7] is complex valued, and you need to consider its real and imaginary 
parts separately. 


A practical limitation of an adaptive antenna array using the LMS algorithm is the dy- 

namic range over which the array can operate. This limitation is due to the fact that the 

speed of response of the weights in the LMS algorithm is proportional to the average 
signal power at the array input. 

(a) Justify the assertion that the dynamic range of average signal power at the array input 
is proportional to R4/f max» where R; is the data rate in b/s and fmax is the maximum 
fade rate in Hz. 

(b) Assuming a proportionality factor of 0.2, by which the ratio Ry/f max is scaled, cal- 
culate the dynamic range of an adaptive antenna array using the LMS algorithm for 
R, = 32 kb/s and fmax = 70 Hz. Comment on your result. (The proportionality factor 
of 0.2 is a reasonable choice for systems using PSK.) 


In this problem we derive the direct matrix inversion algorithm for adjusting the weights 
of an adaptive antenna array. To do so, we revisit the derivation of the Wiener filter 
presented in Chapter 3. 


(a) Show that 
Row = ta 


where R, is an estimate of the correlation matrix of the input vector x[k]: 
K 
R. =$ È xe) 
Kim 


and @,, is an estimate of the cross-correlation vector between x[k] and the reference 
signal d[k]: 


1 K 
ta =X 2 x[k]d* [k] 


The superscript H in the formula for Å, denotes Hermitian transportation (i.e., trans- 
position and complex conjugation), so x[k]x" [k] denotes the outer product of x[k] 
with itself, The summations for both R, and #,, are performed over a total of K 
snapshots, with each snapshot being represented by the pair {x{k], d[k]}. 

(b) Using the formulas of part (a), describe an algorithm for computing the weight vector 
w, given a data set consisting of K snapshots. Hence demonstrate that the complexity 
of this algorithm grows as M? with the size of the weight vector w denoted by M. 





FUNDAMENTAL LIMITS 
IN INFORMATION 
THEORY 


Shannon’s landmark paper on information theory in 1948, and its refinements by other 
researchers, were in direct response to the need of electrical engineers to design 
communication systems that are both efficient and reliable. Efficient communication from 
a source to a user destination is attained through source coding. Reliable communication 
over a noisy channel is attained through error-control coding, This chapter addresses these 
important issues as summarized here: 


> Entropy as the basic measure of information. 
> Source coding theorem and data compaction algorithms. 


> Mutual information and its relation to the capacity of a communication channel for 
information transmission. 


> Channel coding theorem as the basis for reliable communication. 


> Information capacity theorem as the basis for a tradeoff between channel bandwidth and 
signal-to-noise ratio. 


> Rate-distortion theory for source coding with a fidelity criterion. 


i 9.1 Introduction 


As mentioned in the Background and Preview chapter and reiterated along the way, the 
purpose of a communication system is to carry information-bearing baseband signals from 
one place to another over a communication channel. In preceding chapters of the book, 
we have described a variety of modulation schemes for accomplishing this objective. But 
what do we mean by the term information? To address this issue, we need to invoke 
information theory.’ This broadly based mathematical discipline has made fundamental 
contributions, not only to communications, but also to computer science, statistical phys- 
ics, statistical inference, and probability and statistics. 

In the context of communications, information theory deals with mathematical mod- 
eling and analysis of a communication system rather than with physical sources and phys- 
ical channels. In particular, it provides answers to two fundamental questions (among 
others): 


> What is the irreducible complexity below which a signal cannot be compressed? 


> What is the ultimate transmission rate for reliable communication over a noisy 
channel? 
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The answers to these questions lie in the entropy of a source and the capacity of a channel, 
respectively. Entropy is defined in terms of the probabilistic behavior of a source of infor. 
mation; it is so named in deference to the parallel use of this concept in thermodynamics. 
“Capacity is defined as the intrinsic ability of a channel to convey information; it is naturally 
related to the noise characteristics of the channel. A remarkable result that emerges from 
information theory is that if the entropy of the source is less than the capacity of the 
channel, then error-free communication over the channel can be achieved. It is therefore 
befitting that we begin our study of information theory by discussing the relationships 
among uncertainty, information, and entropy. 


| 9.2 Uncertainty, Information, and Entropy 


Suppose that a probabilistic experiment involves the observation of the output emitted by 
a discrete source during every unit of time (signaling interval). The source output is mod- 
eled as a discrete random variable, S, which takes on symbols from a fixed finite alphabet 


f= {So, Sistre Sx-1} (9.1) 
with probabilities 
P(S = s,) = prs k=0,1,...,K-1 (9.2) 


Of course, this set of probabilities must satisfy the condition 


K-1 
x p= 1 (9.3) 
k=0 


We assume that the symbols emitted by the source during successive signaling intervals 
are statistically independent. A source having the properties just described is called a dis- 
crete memoryless source, memoryless in the sense that the symbol emitted at any time is 
independent of previous choices. 

Can we find a measure of how much information is produced by such a source? To 

‘ answer this question, we note that the idea of information is closely related to that of 
uncertainty or surprise, as described next. 

Consider the event S = s+, describing the emission of symbol s, by the source with 
probability p,, as defined in Equation (9.2). Clearly, if the probability p, = 1 and p; = 0 
for all i # k, then there is no “surprise,” and therefore no “information,” when symbol 
S, is emitted, because we know what the message from the source must be. If, on the other 
hand, the source symbols occur with different probabilities, and the probability p, is low, 
then there is more surprise, and therefore information, when symbol s, is emitted by the 
source than when symbol s;, i # k, with higher probability is emitted. Thus, the words 
uncertainty, surprise, and information are all related. Before the event S = s, occurs, there 
is an amount of uncertainty. When the event $ = s occurs there is an amount of surprise. 
After the occurrence of the event S = s,, there is gain in the amount of information, the 
essence of which may be viewed as the resolution of uncertainty. Moreover, the amount 
of information is related to the inverse of the probability of occurrence. 

We define the amount of information gained after observing the event S = s,, which 
occurs with probability p,, as the logarithmic function? 


I(s,) = loe(+) (9.4) 
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This definition exhibits the following important properties that are intuitively satisfying: 
1. 
Hs) = 0 for p, =1 (9.5) 


Obviously, if we are absolutely certain of the outcome of an event, even before it 
occurs, there is #0 information gained. 


Ks) = 0 forO <p, =1 (9.6) 


That is to say, the occurrence of an event S = s, either provides some or no infor- 
mation, but never brings about a loss of information. 


Kse) > Is) for De < p; (2.7) 


That is, the less probable an event is, the more information we gain when it occurs. 
4. Issi) = I(s,) + I(s;) if s, and s; are statistically independent. 


The base of the logarithm in Equation (9.4) is quite arbitrary. Nevertheless, it is the 
standard practice today to use a logarithm to base 2. The resulting unit of information is 
called the bit (a contraction of binary digit). We thus write 


(a) 

loga| — 

Px (9.8) 
-log pk for k = 0, 1,...,K— 1 


Ksk) 


When p, = 1/2, we have I(s,) = 1 bit. Hence, one bit is the amount of information that 
we gain when one of two possible and equally likely (i.e., equiprobable) events occurs. 
Note that the information I(s;) is positive, since the logarithm of a number less than one, 
such as a probability, is negative. 

The amount of information I(s,) produced by the source during an arbitrary signaling 
interval depends on the symbol s, emitted by the source at that time. Indeed, I(s,) is a 
discrete random variable that takes on the values I(so), I(s1), . . . » 1(Sx—1) with probabilities 
Po» Pis +++» Px-1 respectively. The mean of I(s,) over the source alphabet F is given by 


A(P) = E[I(s,)] 


= > pellss) (9.9) 
k=0 


K-1 
=> p loe(*) 

k=0 Pr 
The important quantity H(Y) is called the entropy’ of a discrete memoryless source with 
source alphabet Y. It is a measure of the average information content per source symbol. 
Note that the entropy H(¥) depends only on the probabilities of the symbols in the al- 
phabet & of the source. Thus the symbol F in H(¥) is not an argument of a function but 
rather a label for a source. 
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= SOME PROPERTIES OF ENTROPY 


Consider a discrete memoryless source whose mathematical model is defined by Equations 
(9.1) and (9.2). The entropy H(F) of such a source is bounded as follows: 


0 = H(Y) = log, K (9.10) 


where K is the radix (number of symbols) of the alphabet ¥ of the source. Furthermore, 
we may make two statements: 


1. H(¥) =:0, if and only if the probability p, = 1 for some k, and the remaining 
probabilities in the set are all zero; this lower bound on entropy corresponds to no 
uncertainty. 

2. H(F) = log, K, if and only if p, = 1/K for all k (i.e., all the symbols in the alphabet 
F are equiprobable); this upper bound on entropy corresponds to maximum 
uncertainty. 


To prove these properties of H(¥), we proceed as follows. First, since each proba- 
bility p, is less than or equal to unity, it follows that each term p; log,(1/p,) in Equation 
(9.9) is always nonnegative, and so H(f) = 0. Next, we note that the product term 
pr log2(1/px) is zero if, and only if, pe = 0 or 1. We therefore deduce that H(¥) = 0 if, 
and only if, p, = 0 or 1, that is, p = 1 for some k and all the rest are zero. 

This completes the proofs of the lower bound in Equation (9.10) and statement (1), 

To prove the upper bound in Equation (9.10) and statement (2), we make use of a 
property of the natural logarithm: 


logx=x-1, x20 (9.11) 


This inequality can be readily verified by plotting the functions log x and (x — 1) versus 
x, as shown in Figure 9.1. Here we see that the line y = x — 1 always lies above the curve 
y = log x. The equality holds only at the point x = 1, where the line is tangential to the 
curve, 











FIGURE 9.1 Graphs of the functions x — 1 and log x versus x. 
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To proceed with the proof, consider first any two probability distributions 


(Pos Dis +++» Pr-1} and {qo, qis - - - , Yx—1} on the alphabet F = {so, 51,..., Sk-1} of a 
discrete memoryless source. Then, changing to the natural logarithm, we may write 


= Tk 
2, Pr logo) = log 2 ó S Px lo on( :) 


Hence, using the inequality of Equation (9.11), we get 


© qk 1 of ge 
. = 1 
È, pi ons) log 2 2 n(2 ) 














1 K-1 
= og 2 2 (qe — Pr) 
1 K-1 K-1 
= 4 (24 de S n) = 
We thus have the fundamental inequality 
K-1 
>, Pe los (24) <0 (9.12) 


where the equality holds only if q} = p; for all k. 
Suppose we next put 


n=ż, k=0,1,...,K—-1 (9.13) 


which corresponds to an alphabet ¥ with equiprobable symbols. The entropy of a discrete 
memoryless source with such a characterization equals 


K-1 1 
È qu ls:(4) = log. K (9.14) 
k=0 qk 


Also, the use of Equation (9.13) in Equation (9.12) yields 


K-1 1 
S nie?) «le 
£=0 Pe 


Equivalently, the entropy of a discrete memoryless source with an arbitrary probability 
distribution for the symbols of its alphabet ¥ is bounded as 


H(Y) = log, K 


Thus H(P) is always less than or equal to log, K. The equality holds only if the symbols 
in the alphabet F are equiprobable, as in Equation (9.13). This completes the proof of 
Equation (9.10) and statements (1) and (2). 


> ExaMPLe 9.1 Entropy of Binary Memoryless Source 


To illustrate the properties of H(¥), we consider a binary source for which symbol 0 occurs 
with probability pọ and symbol 1 with probability p4 = 1 — po. We assume that the source 
is memoryless so that successive symbols emitted by the source are statistically independent. 
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FIGURE 9.2 Entropy function #(po). 


The entropy of such a source equals 


H(P) = —po logs po — pi logs pı 


5 9.15 
—Po logs Po — (1 — po) loga(1 — po) bits (2.15) 


ll 


from which we observe the following: 


1. When po = 0, the entropy H() = 0; this follows from the fact that x log x > 0 as 
x0. 

2. When po = 1, the entropy H(Y) = 0. 

3, The entropy H(¥) attains its maximum value, Hway = 1 bit, when pi = po = 1/2, that 
is, symbols 1 and 0 are equally probable. 


The function of po given on the right-hand side of Equation (9.15) is frequently en- 
countered in information-theoretic problems. It is therefore customary to assign a special 
symbol to this function. Specifically, we define 


3€(Po) = —Po logs po — (1 — po) log2(1 — po) (9.16) 


We refer to (po) as the entropy function. The distinction between Equation (9.15) and Equa- 
tion (9.16) should be carefully noted. The H(f) of Equation (9.15) gives the entropy of 
a discrete memoryless source with source alphabet F. The %€(po) of Equation (9.16), on 
the other hand, is a function of the prior probability po defined on the interval [0, 1]. Accord- 
ingly, we may plot the entropy function #(po) versus po, defined on the interval [0, 1], as 
in Figure 9.2. The curve in Figure 9.2 highlights the observations made under points 1, a 
and 3. 


EXTENSION OF A DISCRETE MEMORYLESS SOURCE 


In discussing information-theoretic concepts, we often find it useful to consider blocks 
rather than individual symbols, with each block consisting of # successive source symbols. 
‘We may view each such block as being produced by an extended source with a source 
alphabet ¥” that has K” distinct blocks, where K is the number of distinct symbols in the 
source alphabet F of the original source. In the case of a discrete memoryless source, the 
source symbols are statistically independent. Hence, the probability of a source symbol in 
F” is equal to the product of the probabilities of the » source symbols in ¥ constituting 
the particular source symbol in Y”. We may thus intuitively expect that H(¥”), the entropy 
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of the extended source, is equal to » times H(Y), the entropy of the original source. That 
is, We may write : 


H(G”) = nH(S) (9.17) 


> EXAMPLE 9.2 Entropy of Extended Source 


Consider a discrete memoryless source with source alphabet F = {so, s4, s2} with respective 


probabilities 
Po = 4 
pi =4 
Po = 


Hence, the use of Equation (9.9) yields the entropy of the source as 


1 1 1 
H(P) = po tos(4) + pr ls:(+) + po tor) 


1 1 1 
= log2(4) + 4 log2(4) + 3 log,(2) 





Consider next the second-order extension of the source. With the source alphabet 9 
consisting of three symbols, it follows that the source alphabet ¥? of the extended source has 
nine symbols. The first row of Table 9.1 presents the nine symbols of $2, denoted as ap, 
1,..., og. The second row of the table presents the composition of these nine symbols in 
terms of the corresponding sequences of source symbols so, s4, and sz, taken two at a time. 
The probabilities of the nine source symbols of the extended source are presented in the last 
row of the table. Accordingly, the use of Equation (9.9) yields the entropy of the extended 
source as 


8 
1 
2) ers 
HS?) = > pla) logs re 
sos 16 pti 16+ 1] 8 pA 16 
= i6 og2(16) 16 og2( 8 log2(8) 16 og2(16) 


1 1 1 1 1 
+ Té loga(16) + 3 log,(8) + F log,(8) + 3 log.(8) + 3 log2(4) 


= 3 bits 
We thus see that H(¥?) = 2H() in accordance with Equation (9.17). < 

TABLE 9.1 Alphabet particulars of second-order extension 

of a discrete memoryless source 
Symbols of F? To T Tz T Tg T5 Ts T Tg 
Corresponding sequences SoSo Sos1 S0S2 $150 S484 $18 S250 $254 S982 

of symbols of F 
Probability p(0;), EO Oe 3 w i a a i 2 


#=0,1,...,8 
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| 9.3 Source-Coding Theorem 


An important problem in communications is the efficient representation of data generated 
by a discrete source. The process by which this representation is accomplished is called 
source encoding. The device that performs the representation is called a source encoder, 
For the source encoder to be efficient, we require knowledge of the statistics of the source, 
In particular, if some source symbols are known to be more probable than others, then 
we may exploit this feature in the generation of a source code by assigning short code 
words to frequent source symbols, and Jong code words to rare source symbols. We refer 
to such a source code as a variable-length code. The Morse code is an example of a variable- 
length code. In the Morse code, the letters of the alphabet and numerals are encoded into 
streams of marks and spaces, denoted as dots “.” and dashes “-”, respectively. In the 
English language, the letter E occurs more frequently than the letter Q, for example, so 
the Morse code encodes E into a single dot “.”, the shortest code word in the code, and 
it encodes Q into “- - . -”, the longest code word in the code. 

Our primary itetest: is in the development of an efficient source encoder that satisfies 
two functional requirements: i 


1. The code words produced by the encoder are in binary form. 


2. The source code is uniquely decodable, so that the original source sequence can be 
reconstructed perfectly from the encoded binary sequence. ; 


Consider then the scheme shown in Figure 9.3, which depicts a discrete memoryless 
source whose output s, is converted by the source encoder into a block of Os and 1s, 
denoted by b,. We assume that the source has an alphabet with K different symbols, and 
that the kth symbol s, occurs with probability p,, k = 0, 1,..., K — 1. Let the binary 
code word assigned to symbol s, by the encoder have length /,,, measured in bits. We define 
the average code-word length, L, of the source encoder as 

K-1 


Ts Z bala (9.18) 


In physical terms, the parameter L represents the average number of bits per source symbol 
used in the source encoding process. Let Lmin denote the minimum possible value of L. 
We then define the coding efficiency of the source encoder as 


L. 

=o 9.19 

n T (9.19) 

With L = Lmin, we clearly have 7 = 1. The source encoder is said to be efficient when n 


approaches unity. 

But how is the minimum value Lmin determined? The answer to this fundamental 
question is embodied in Shannon’s first theorem: the source-coding theorem,‘ which may 
be stated as follows: 


Given a discrete memoryless source of entropy H(FP), the average code-word length 
L for any distortionless source encoding scheme is bounded as 


L= HQ) (9.20) 





Discrete SE 
memoryless 
source 


b, A 
Source x Binary 
encoder sequence 














FIGURE 9.3 Source encoding. 
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(A proof of this theorem for a particular class of source codes is presented in the next 
section.) According to the source-coding theorem, the entropy H(ẸF) represents a funda- 
mental limit on the average number of bits per source symbol necessary to represent a 
discrete memoryless source in that it can be made as small as, but no smaller than, the 
entropy H(F). Thus with Lmin = H(¥), we may rewrite the efficiency of a source encoder 
in terms of the entropy H(Y) as 


AS) 


DA (9.21) 


n= 


j 9.4 Data Compaction 


A common characteristic of signals generated by physical sources is that, in their natural 
form, they contain a significant amount of information that is redundant, the transmission 
of which is therefore wasteful of primary communication resources. For efficient signal 
transmission, the redundant information should be removed from the signal prior to trans- 
mission. This operation, with no loss of information, is ordinarily performed on a signal 
in digital form, in which case we refer to it as data compaction or lossless data compression. 
The code resulting from such an operation provides a representation of the source output 
that is not only efficient in terms of the average number of bits per symbol but also exact 
in the sense that the original data can be reconstructed with no loss of information. The 
entropy of the source establishes the fundamental limit on the removal of redundancy from 
the data. Basically, data compaction is achieved by assigning short descriptions to the most 
frequent outcomes of the source output and longer descriptions to the less frequent ones. 

In this section, we discuss some source-coding schemes for data compaction. We 
begin the discussion by describing a type of source code known as a prefix code, which is 
not only decodable but also offers the possibility of realizing an average code-word length 
that can be made arbitrarily close to the source entropy. 


PREFIX CODING 


Consider a discrete memoryless source of alphabet {so s;,..., Sx—1} and statistics 
{Pos Pis <--> Px—1}. For a source code representing the output of this source to be of 
practical use, the code has to be uniquely decodable. This restriction ensures that for each 
finite sequence of symbols emitted by the source, the corresponding sequence of code words 
is different from the sequence of code words corresponding to any other source sequence. 
We are specifically interested in a special class of codes satisfying a restriction known 
as the prefix condition. To define the prefix condition, let the code word assigned to 
source symbol s, be denoted by (Mk, Mk -.., Mg), Where the individual elements 
Mr, +++, My, are Os and 1s, and 7 is the code-word length. The initial part of the code 
word is represented by the elements Mey +. +5 My, for some i = n. Any sequence made up 
of the initial part of the code word is called a prefix of the code word. A prefix code is 
defined as a code in which no code word is the prefix of any other code word. 

To illustrate the meaning of a prefix code, consider the three source codes described 
in Table 9.2. Code I is not a prefix code since the bit 0, the code word for so, is a prefix 
of 00, the code word for s2. Likewise, the bit 1, the code word for s,, is a prefix of 11, the 
code word for sz. Similarly, we may show that code HI is not a prefix code, but code II is. 

To decode a sequence of code words generated froma prefix source code, the source 
decoder simply starts at the beginning of the sequence and decodes one code word at a 
time. Specifically, it sets up what is equivalent to a decision tree, which is a graphical 
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i TABLE 9.2 Illustrating the definition of a prefix code 





Source Symbol Probability of Occurrence Code I Code IT Code Itt 
so 0.5 0 0 0 
s 0.25 1 10 01 
s 0.125 00 110 011 
s 0.125 11 111 0111 





portrayal of the code words in the particular source code. For example, Figure 9.4 depicts 
the decision tree corresponding to code II in Table 9.2. The tree has an initial state and 
four terminal states corresponding to source symbols so, S1, $2, and sz. The decoder always 
starts at the initial state. The first received bit moves the decoder to the terminal state s, 
if it is 0, or else to a second decision point if it is 1. In the latter case, the second bit moves 
the decoder one step further down the tree, either to terminal state s; if it is 0, or else to 
a third decision point if it is 1, and so on. Once each terminal state emits its symbol, the 
decoder is reset to its initial state. Note also that each bit in the received encoded sequence 
is examined only once. For example, the encoded sequence 1011111000 ... is readily 
decoded as the source sequence s153S2SoSp.... The reader is invited to carry out this 
decoding. ` . 

A prefix code has the important property that it is always uniquely decodable. But 
the converse is not necessarily true. For example, code M in Table 9.2 does not satisfy the 
prefix condition, yet it is uniquely decodable since the bit 0 indicates the beginning of each 
code word in the code. 

Moreover, if a prefix code has been constructed for a discrete memoryless source 
with source alphabet {so, S1, - - - , Sk-1} and source statistics {Po, f1,..+5 Px-1} and the 
code word for symbol s, has length J,, k = 0,1,..., K — 1, then the code-word lengths 
of the code always satisfy a certain inequality known as the Kraft-McMillan Inequality,’ 
as shown by 


2k <1 (9.22) 


Initial 
state 





FIGURE 9.4 Decision tree for code II of Table 9.2. 
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where the factor 2 refers to the radix (number of symbols) in the binary alphabet. It is 
important to note, however, that the Kraft-McMillan inequality does not tell us that a 
source code is a prefix code. Rather, it is merely a condition on the code-word lengths of 
the code and not on the code words themselves. For example, referring to the three codes 
listed in Table 9.2, we note the following: 


> Code I violates the Kraft-McMillan inequality; it cannot therefore be a prefix code. 
> The Kraft-McMillan inequality is satisfied by both codes II and IH; but only code II 
is a prefix code. 


Prefix codes are distinguished from other uniquely decodable codes by the fact that 
the end of a code word is always recognizable. Hence, the decoding of a prefix can be 
accomplished as soon as the binary sequence representing a source symbol is fully received. 
For this reason, prefix codes are also referred to as instantaneous codes. 

Given a discrete memoryless source of entropy H(Y), a prefix code can be constructed 
with an average code-word length L, which is bounded as follows: 


HY) = L< HP) +1 (9.23) 

The left-hand bound of Equation (9.23) is satisfied with equality under the condition that 
symbol s, is emitted by the source with probability 

De = 27% (9.24) 


where J, is the length of the code word assigned to source symbol s,. We then have 


K-1 


Sr- Sant 


Under this condition, the Kraft-McMillan inequality of Equation (9.22) tells us that we 
can construct a prefix code, such that the length of the code word assigned to source symbol 
Sk is —logap,. For such a code, the average code-word length is 


= K-i l 
E=, ai (9.25) 
k=0 


and the corresponding entropy of the source is 


H(t) = > + logs(2') 


K-1 pi 


(9.26) 


li 
iM? 
FF 


Hence, in this special (rather meretricious) case, we find from Equations (9.25) and (9.26) 
that the prefix code is matched to the source in that L = H(). 

But how do we match the prefix code to an arbitrary discrete memoryless source? 
The answer to this problem lies in the use of an extended code. Let L,, denote the e average 
code-word length of the extended prefix code. For a uniquely decodable code, L,, is the 
smallest possible. From Equation (9.23), we deduce that 


HIS”) = L, < HF") +1 (9.27) ° 
Substituting Equation (9.17) for an extended source into Equation (9.27), we get 
nH(P) = Ly < nH) +1 
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or, equivalently, 


L 1 
H(Y) = A < HY) + = (9.28) 


In the limit, as n approaches infinity, the lower and upper bounds in Equation (9.28) 
converge, as shown by 


ale 


lim 


nwo 


L, = HP) (9.29) 


We may therefore state that by making the order # of an extended prefix sonce 
encoder large enough, we can make the code faithfully represent the discrete memoryless 
source Y as closely as desired. In other words, the average code-word length of an extended 
prefix code can be made as small as the entropy of the source provided the extended code 
has a high enough order, in accordance with the source-coding theorem. However, the 
price we have to pay for decreasing the average code-word length is increased decoding 
complexity, which is brought about by the high order of the extended prefix code. 


= HUFFMAN CODING 


We next describe an important class of prefix codes known as Huffman codes. The basic 
idea behind Huffman coding’ is to assign to each symbol of an alphabet a sequence of bits 
roughly equal in length to the amount of information conveyed by the symbol in question. 
The end result is a source code whose average code-word length approaches the funda- 
mental limit set by the entropy of a discrete memoryless source, namely, H(). The essence , 
of the algorithm used to synthesize the Huffman code is to replace the prescribed set of 
soutce statistics of a discrete memoryless source with a simpler one. This reduction process 
is continued in a step-by-step manner until we are left with a final set of only two source 
statistics (symbols), for which (0, 1) is an optimal code. Starting from this trivial code, we 
then work backward and thereby construct the Huffman code for the given source. 
Specifically, the Huffman encoding algorithm proceeds as follows: 


1. The source symbols are listed in order of decreasing probability. The two source 
symbols of lowest probability are assigned a 0 and a 1. This part of the step is referred 
to as a splitting stage. 

2. These two source symbols are regarded as being combined into a new source symbol 
with probability equal to the sum of the two original probabilities. (The list of source 
symbols, and therefore source statistics, is thereby reduced in size by one.) The prob- 
ability of the new symbol is placed in the list in accordance with its value. 

3. The procedure is repeated until we are left with a final list of source statistics (sym- 
bols) of only two for which a 0 and a 1 are assigned. 


The code for each (original) source symbol is found by working backward and tracing the 
sequence of 0s and 1s assigned to that symbol as well as its successors. 


> EXAMPLE 9.3 Huffman Tree 


The five symbols of the alphabet of a discrete memoryless source and their probabilities are 
shown in the two leftmost columns of Figure 9.54. Following through the Huffman algorithm, 
we reach the end of the computation in four steps, resulting in the Huffman tree shown in 
Figure 9.5a, The code words of the Huffman code for the source are tabulated in Figure 9,50. 
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Symbol Stage | Stage II Stage [ll Stage IV Symbol Probability Code word 
0.4 00 
s 0.4 0.4 0.4 0.6 È 2 mes 16 
o a 32 0.2 11 
s 0.2 0.2 0.4 0.4 “ 01 010 
ie) 1 0.1 011 
s2 0.2 0.2 0.2 "e 
0 1 (b) 
Ig 0.1 0.2 
1 
84 0.1 


(a) 
FIGURE 9.5 (a) Example of the Huffman encoding algorithm. (b) Source code. 


The average code-word length is therefore 
L = 0.4(2) + 0.2(2) + 0.2(2) + 0.1(3) + 0.1(3) 
= 2.2 


The entropy of the specified discrete memoryless source is calculated as follows [see 
Equation (9.9)]: 


HI?) = 0.4 tos) +02 (35) +02 toe:( 4) 


1 1 
+ 0.1 te.) + 0.1 ioga) 
= 0.52877 + 0.46439 + 0.46439 + 0.33219 + 0.33219 


= 2.12193 bits 
For the example at hand, we may make two observations: 


1. The average code-word length L exceeds the entropy H(¥) by only 3.67 percent. 
2. The average code-word length L does indeed satisfy Equation (9.23). q 


It is noteworthy that the Huffman encoding process (i.e., the Huffman tree) is not 
unique. In particular, we may cite two variations in the process that are responsible for 
the nonuniqueness of the Huffman code. First, at each splitting stage in the construction 
of a Huffman code, there is arbitrariness in the way a 0 and a 1 are assigned to the last 
two source symbols. Whichever way the assignments are made, however, the resulting 
differences are trivial. Second, ambiguity arises when the probability of a combined symbol 
(obtained by adding the last two probabilities pertinent to a particular step) is found to 
equal another probability in the list. We may proceed by placing the probability of the 
new symbol as high as possible, as in Example 9.3. Alternatively, we may place it as low 
as possible. (It is presumed that whichever way the placement is made, high or low, it is 
consistently adhered to throughout the encoding process.) But this time, noticeable differ- 
ences arise in that the code words in the resulting source code can have different lengths. 
Nevertheless, the average code-word length remains the same. 

As a measure of the variability in code-word lengths of a source code, we define the 
variance of the average code-word length L over the ensemble of source symbols as 


K-1 
= > Pelle - TP (9.30) 


where po, Pis - -> Px-1 are the source statistics, and /, is the length of the code word 
assigned to source symbol sg. It is usually found that when a combined symbol is moved 
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as high as possible, the resulting Huffman code has a significantly smaller variance a° than 
when it is moved as low as possible. On this basis, it is reasonable to choose the former 
Huffman code over the latter. 


LEMPEL—ZIV CODING 


A drawback of the Huffman code is that it requires knowledge of a probabilistic model 
of the source; unfortunately, in practice, source statistics are not always known a priori, 
Moreover, in modeling text we find that storage requirements prevent the Huffman code 
from capturing the higher-order relationships between words and phrases, thereby com- 
promising the efficiency of the code. To overcome these practical limitations, we may use 
the Lempel-Ziv algorithm,’ which is intrinsically adaptive and simpler to implement than 
Huffman coding. 

Basically, encoding in the Lempel-Ziv algorithm is accomplished by parsing the 
source data stream into segments that are the shortest subsequences not encountered pre- 
viously. To illustrate this simple yet elegant idea, consider the example of an input binary 
sequence specified as follows: 


000101110010100101 ... 


It is assumed that the binary symbols 0 and 1 are already stored in that order in the code 
book. We thus write 


Subsequences stored: 0,1 
Data to be parsed: 000101110010100101 ... 


The encoding process begins at the left. With symbols 0 and 1 already stored, the shortest 
subsequence of the data stream encountered for the first time and not seen before is 00; 
so we write 


Subsequences stored: 0, 1, 00 
Data to be parsed: 0101110010100101 ... 


The second shortest subsequence not seen before is 01; accordingly, we go on to write 


Subsequences stored: 0, 1, 00, 01 
Data to be parsed: 01110010100101... 


The next shortest subsequence not encountered previously is 011; hence, we write 


Subsequences stored: 0, 1, 00, 01, 011 
Data to be parsed: 10010100101 ... 
We continue in the manner described here until the given data stream has been completely 


parsed. Thus, for the example at hand, we get the code book of binary subsequences shown 
in the second row of Figure 9.6. 


Numerical positions: 1 2 3 4 5 6 J 8 9 
Subsequences: 0 1 00 01 011 10 010 100 1% 
Numerical representations: 11 12 42 21 41 61 62 
Binary encoded blocks: 0010 0011 1001 0100 1000 1100 1101 


FIGURE 9.6 [Illustrating the encoding process performed by the Lempel-Ziv algorithm on the 
binary sequence 000101110010100101.... 
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The first row shown in this figure merely indicates the numerical positions of the 
individual subsequences in the code book. We now recognize that the first subsequence of 
the data stream, 00, is made up of the concatenation of the first code book entry, 0, with 
itself; it is therefore represented by the number 11. The second subsequence of the data 
stream, 01, consists of the first code book entry, 0, concatenated with the second code 
book entry, 1; it is therefore represented by the number 12. The remaining subsequences 
are treated in a similar fashion. The complete set of numerical representations for the 
various subsequences in the code book is shown in the third row of Figure 9.6. As a further 
example illustrating the composition of this row, we note that the subsequence 010 consists 
of the concatenation of the subsequence 01 in position 4 and symbol 0 in position 1; hence, 
the numerical representation 41. The last row shown in Figure 9.6 is the binary encoded 
representation of the different subsequences of the data stream. 

The last symbol of each subsequence in the code book (i.e., the second row of Figure 
9.6) is an innovation symbol, which is so called in recognition of the fact that its appendage 
to a particular subsequence distinguishes it from all previous subsequences stored in the 
code book. Correspondingly, the last bit of each uniform block of bits in the binary en- 
coded representation of the data stream (i.e., the fourth row in Figure 9.6) represents the 
innovation symbol for the particular subsequence under consideration. The remaining bits 
provide the equivalent binary representation of the “pointer” to the root subsequence that 
matches the one in question except for the innovation symbol. 

The decoder is just as simple as the encoder. Specifically, it uses the pointer to identify 
the root subsequence and then appends the innovation symbol. Consider, for example, the 
binary encoded block 1101 in position 9. The last bit, 1, is the innovation symbol. The 
remaining bits, 110, point to the root subsequence 10 in position 6. Hence, the block 1101 
is decoded into 101, which is correct. 

From the example described here, we note that, in contrast to Huffman coding, the 
Lempel—Ziv algorithm uses fixed-length codes to represent a variable number of source 
symbols; this feature makes the Lempel—Ziv code suitable for synchronous transmission. 
In practice, fixed blocks of 12 bits long are used, which implies a code book of 4096 
entries. 

For a long time, Huffman coding was unchallenged as the algorithm of choice for 
data compaction. However, the Lempel—Ziv algorithm has taken over almost completely 
from the Huffman algorithm. The Lempel—Ziv algorithm is now the standard algorithm 
for file compression, When it is applied to ordinary English text, the Lempel—Ziv algorithm 
achieves a compaction of approximately 55 percent. This is to be contrasted with a com- 
paction of approximately 43 percent achieved with Huffman coding. The reason for this 
behavior is that, as mentioned previously, Huffman coding does not take advantage of the 
intercharacter redundancies of the language. On the other hand, the Lempel—Ziv algorithm 
is able to do the best possible compaction of text (within certain limits) by working effec- 
tively at higher levels. 


| 9.5 Discrete Memoryless Channels 


Up to this point in the chapter, we have been preoccupied with discrete memoryless sources 
responsible for information generation. We next consider the issue of information trans- 
mission, with particular emphasis on reliability. We start the discussion by considering a 
discrete memoryless channel, the counterpart of a discrete memoryless source. 

A discrete memoryless channel is a statistical model with an input X and an output 
Y that is a noisy version of X; both X and Y are random variables. Every unit of time, the 
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FIGURE 9.7 Discrete memoryless channel. 


channel accepts an input symbol X selected from an alphabet % and, in response, it emits 
an output symbol Y from an alphabet Y. The channel is said to be “discrete” when both 
of the alphabets ¥ and Y have finite sizes. It is said to be “memoryless” when the current 
output symbol depends only on the current input symbol and not any of the previous ones, 

Figure 9.7 depicts a view of a discrete memoryless channel. The channel is described 
in terms of an input alphabet 


Æ = (xo, Xis - -+ s Xy—1}5 (9.31) 
an output alphabet, 
l Y = {Yo Yis -> -> ¥x-a)s (9.32) 
and a set of transition probabilities 
plys|x) = P(Y = y| X =x) for all j and k (9.33) 
Naturally, we have 
0<ply,|x) 51 forall j and k (9.34) 


Also, the input alphabet Æ and output alphabet Y need not have the same size. For 
example, in channel coding, the size K of the output alphabet Y may be larger than the 
size J of the input alphabet %; thus, K = J. On the other hand, we may have a situation 
in which the channel emits the same symbol when either one of two input symbols is sent, 
in which case we have K = J. 

A convenient way of describing a discrete memoryless channel is to arrange the 
various transition probabilities of the channel in the form of a matrix as follows: 


b(¥0| Xo) p(y lxo) aeg P(¥x-1|*0) 
P= P(yo|%1) piyi |) i P(Yx-1 |a) (9.35) 
P(Yo |)-1) Ply [xy-1) aa px- | xy-1) 


The J-by-K matrix P is called the channel matrix, or transition matrix. Note that each row 
of the channel matrix P corresponds to a fixed channel input, whereas each column of the 
matrix corresponds to a fixed channel output. Note also that a fundamental property of 
the channel matrix P, as defined here, is that the sum of the elements along any row of the 
matrix is always equal to one; that is, 


K-1 


S plvelx) =1  forallj (9.36) 
k=0 
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Suppose now that the inputs to a discrete memoryless channel are selected according 
to the probability distribution {p(x;), į = 0, 1,..., J — 1}. In other words, the event that 
the channel input X = x, occurs with probability 


pix) =P(X=x) forj=0,1,...,J-1 (9,37) 


Having specified the random variable X denoting the channel input, we may now specify 
the second random variable Y denoting the channel output. The joint probability distri- 
bution of the random variables X and Y is given by 


D(X js Ye) = P(X = x; Y = ye) 
= P(Y = y,|X = x) P(X = x) (9.38) 
= p(ve|x,)p(x,) 
The marginal probability distribution of the output random variable Y is obtained by 


averaging out the dependence of p(x;, Yg) on x;, as shown by 


ply) = P(Y = ya) 
J-1 
= > P(Y = »,|X = x,)P(X = x) (9.39) 
£ 





J-1 
= > blyklxpl) fork =0,1,...,K-1 
= 


The probabilities p(x;) for j = 0, 1, . . . , J — 1, are known as the a priori probabilities 
of the various input symbols. Equation (9.39) states that if we are given the input a pri- 
ori probabilities p(x;) and the channel matrix [i.e., the matrix of transition probabilities 
(ye |%,)], then we may calculate the probabilities of the various output symbols, the p(y,). 


& EXAMPLE 9.4 Binary Symmetric Channel 


The binary symmetric channel is of great theoretical interest and practical importance. It is a 
special case of the discrete memoryless channel with J = K = 2. The channel has two input 
symbols (xo = 0, x; = 1) and two output symbols (yy = 0, y4 = 1). The channel is symmetric 
because the probability of receiving a 1 if a 0 is sent is the same as the probability of receiving 
a 0 if a 1 is sent. This conditional probability of error is denoted by p. The transition prob- 


ability diagram of a binary symmetric channel is as shown in Figure 9.8. 4 
l-p 
Xg=0 ¥o= 0 
P 
Pp 
x=l ip yal 


Ficure 9.8 - Transition probability diagram of binary symmetric channel. 
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It is of interest to relate the transition probability diagram of Figure 9.8 to the con. 
ditional probabilities of error p19 and po, that were determined for the PCM receiver jn 
Section 3.3. For the case when the binary symbols 0 and 1 are equiprobable, we showed 
that the optimized values of these two error probabilities are equal. Indeed, recalling the 
following definitions (using the terminology of Figure 9.8): i 


Pio = Ply = 1|x = 0) 
and 

Por = Ply = O|x = 1) 
we immediately see that for the PCM receiver of Figure 3.4: 


Pio = Po F P 


| 9.6 Mutual Information 


Given that we think of the channel output Y (selected from alphabet Y) as a noisy version 
of the channel input X (selected from alphabet 2), and that the entropy H(2) is a measure 
of the prior uncertainty about X, how can we measure the uncertainty about X after 
observing Y? To answer this question, we extend the ideas developed in Section 9,2 by 
defining the conditional entropy of X selected from alphabet %, given that Y = yg. Spe- 
cifically, we write 


J-i 1 : 
H(Æ| Y = yx) = > plx; lya) logs Ear (9.40) 


This quantity is itself a random variable that takes on the values 
H(®|Y = Yo), ---, H(Æ|Y = yx-1) with probabilities p(yo), . - +5 P(¥x—1), respectively. 
The mean of entropy H(Æ| Y = y,) over the output alphabet Y is therefore given by 


K-1 
H(%|%) = > H(X |Y = yap lya) 
K-1 J-1 1 
= j ——_ AL 
25 pil ydp (ye) oe | (9.41) 
K-1J-1 


= > > plap ya) lo Fam 
Mo fo” sare a p(x;| Ye) 


where, in the last line, we have made use of the relation 


P(x js ya) = pal ¥e)P( ye) (9.42) 


The quantity H(%|¥) is called a conditional entropy. It represents the amount of uncer- 
tainty remaining about the channel input after the channel output has been observed. 
Since the entropy H() represents our uncertainty about the channel input before 
observing the channel output, and the conditional entropy Hi (Æ |Y) represents our uncet- 
tainty about the channel input after observing the channel output, it follows that the 
difference H(Æ) — H(X |Y) must represent our uncertainty about the channel input that is 
resolved by observing the channel output. This important quantity is called the mutual 
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information of the channel. Denoting the mutual information by I(%; Y), we may thus 
write 

(Z; Y) = — HZ |Y) (9.43) 
Similarly, we may write 
KY; X) = HY) — HY |2) (9.44) 


where H(%) is the entropy of the channel output and H(Y |Æ) is the conditional entropy 
of the channel output given the channel input. 


PROPERTIES OF MUTUAL INFORMATION 


The mutual inforamtion 1(%; Y) has the following important properties. 


Property 1 
The mutual information of a channel is symmetric; that is 


Z; Y) = KY; X) (9.45) 


where the mutual information I(%; Y) is a measure of the uncertainty about the channel 
input that is resolved by observing the channel output, and the mutual information 

(Y; 2) is a measure of the uncertainty about the channel output that is resolved by sending 
the channel input. 


To prove this property, we first use the formula for entropy and then use Equations 
(9.36) and (9.38), in that order, to express H(%) as 


= 
l 


| 1 
H£) = 2, plx x) owe ral 


j=0 
z he 
= 2 p(x) daca È, plyel x) (9.46) 
J-i K- 1 
a ERA piya |x; p(x;) logs Fal 
J-1 K-1 


1l 
ME 


1 
, =, P(X) Ya) e|- 


Hence, substituting Equations (9.41) and (9.46) into Equation (9.43) and then combining 
terms, we obtain 





for (xl ye) 
= 2 2 D(x}, Ye) los. Ë a | (9.47) 


From Bayes’ rule for conditional probabilities, we have [see Equations (9.38) and (9.42)] 


plx; | ye) hy plyel x) 
plx) pliye) 





(9.48) 
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Hence, substituting Equation (9.48) into Equation (9.47) and interchanging the order of 
summation, we may write 


K-1J-1 i 
15%) = OD pla) lo? zw] 





k=0 j=0 blye) (9 
49 
= KY; &) ) 
which is the desired result. 
Property 2 
The mutual information is always nonnegative; that is 
KZ; Y) = 0 (9.50) 
To prove this property, we first note from Equation (9.42) that 
p (xj, Ya) 
xj pales aaa 9, 
(x;| Ye) A) (9.51) 


Hence, substituting Equation (9.51) into Equation (9.47), we may express the mutual 
information of the channel as 


D(X;s yn) ) 
3 Been LAET ae 


Next, a direct application of the sine inequality [defined by Equation (9.12)] yields 
the desired result 


wet 


1(%; Y) = 0 
with equality if, and only if, 
D(x Ye) = P(x)P(y2) for all j and k (9.53) 


Property 2 states that we cannot lose information, on the average, by observing the 
output of a channel. Moreover, the mutual information is zero if, and only if, the input 
and output symbols of the channel are statistically independent, as in Equation (9.53). 


Property 3 
The mutual information of a channel is related to the joint entropy of the channel input 
and channel output by 


KX; Y) = H(X) + H(Y) — H(z, Y) (9.54) 
where the joint entropy H(X, Y) is defined by 
J-1 K-1 1 
+ Ya) lo (9.55) 
“2% 2 POE ela Xj 5) 
To prove Equation (9.54), we first rewrite the definition for the joint entropy 


H(X, Y) as 
J-1 K-i 


H(Z, a3 Xp D(x}. Ya) ) ozo] 
nae 


eenei 
Plx; Ye) 


K-1 1 
2 By Plt m0) ) 1o | 


i 


(9.56) 


whe 
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FIGURE 9.9 Illustrating the relations among various channel entropies. 


The first double summation term on the right-hand side of Equation (9.56) is recognized 
as the negative of the mutual information of the channel, I(%; Y), previously given in 
Equation (9.52), As for the second summation term, we manipulate it as follows: 


Jnl K-14 1 Jer 1 K-1 
> > P(X js Ya) lol i => oel] 2 D(x Ye) 


j=0 k=0 j=0 p(x) 


K~1 1 J-1 
+ 5 log|—— 3 
S tos 5) S ven vo 


Jot 1 
= > p(x;) g| -+ (9.57) 


K-1 1 
l ecu 
= 2, Ply) eed | 


= H(#) + H(%) 
Accordingly, using Equations (9.52) and (9.57) in Equation (9.56), we get the result 
H(X, Y) = KX; Y) + Hie) + H) (9.58) 


Rearranging terms in this equation, we get the result given in Equation (9.54), thereby 
confirming Property 3. 

We conclude our discussion of the mutual information of a channel by providing a 
diagramatic interpretation of Equations (9.43), (9.44), and (9.54). The interpretation is 
given in Figure 9.9. The entropy of channel input X is represented by the circle on the left. 
The entropy of channel output Y is represented by the circle on the right. The mutual 
information of the channel is represented by the overlap between these two circles. 


| 9.7 Channel Capacity 





Consider a discrete memoryless channel with input alphabet Æ, output alphabet Y, and 
transition probabilities p(y, | x;), where j = 0, 1,...,J— 1and k = 0, 1, ... , K — 1. The 
mutual information of the channel is defined by the first line of Equation (9.49), which is 


reproduced here for convenience: 
J-1 
P(yelx;) 
> D(X Ya) oe —— 





KX; Y) = 2 


0 j=0 


Pliye) 
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Here we note that [see Equation (9.38)] 
p(x, Ya) = (ye | x,)p(x;) 


Also, from Equation (9.39), we have 


joi 
Pye) = > piye |x) p(x,) 


From these three equations we see that it is necessary for us to know the input probability 
distribution {p(x;) |} = 0, 1,..., J — 1} so that we may calculate the mutual information 
(2; Y). The mutual information of a channel therefore depends not only on the channel 
but also on the way in which the channel is used. 
The input probability distribution {p(x;)} is obviously independent of the channel, . 

We can then maximize the mutual information I(%; Y) of the channel with respect to 
{p(x;)}. Hence, we define the channel capacity of a discrete memoryless channel as the 
maximum mutual information I(%; Y) in any single use of the channel (i.e., signaling 
interval), where the maximization is over all possible input probability distributions {p(x;)} 
on &. The channel capacity is commonly denoted by C. We thus write 


C = max I(®; Y) (9.59) 
{p(x} 


The channel capacity C is measured in bits per channel use, or bits per transmission. 
Note that the channel capacity C is a function only of the transition probabilities 
P(¥z|x;), which define the channel. The calculation of C involves maximization of the 
mutual information I(%; Y) over J variables [i.e., the input probabilities p(x9), . . . ,(%)—1)] 
subject to two constraints: 


p(x;) = 0 for all j 


and 


J-1 
> p(x) =1 


In general, the variational problem of finding the channel capacity C is a challenging task. 


> EXAMPLE 9.5 Binary Symmetric Channel (Revisited) 


Consider again the binary symmetric channel, which is described by the transition probability 
diagram of Figure 9.8. This diagram is uniquely defined by the conditional probability of error 


p. 

The entropy H(X) is maximized when the channel input probability p(xo) = p(x;) = 
1/2, where xo and x, are each 0 or 1. The mutual information I(#; Y) is similarly maximized, 
so that we may write 


C = Ke; Y) | ptep)=pla)=1/2 
From Figure 9.8, we have 

plyolx) = piyi lxo) = p 
and 


Plyolxo) = ply lx) =1 -p 
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FIGURE 9.10 Variation of channel capacity of a binary symmetric channel with transition proba- 
bility p. 


Therefore, substituting these channel transition probabilities into Equation (9.49) with 
J = K = 2, and then setting the input probability p(x 9) = p(x,) in accordance with Equation 
(9.59), we find that the capacity of the binary symmetric channel is 


C=1+ p log, p + (1 — p) log:(1 — p) (9.60) 
Using the definition of the entropy function given in Equation (9.16), we may reduce Equation 
(9.60) to 
C=1- Hip) 


The channel capacity C varies with the probability of error (transition probability) p in 
a convex manner as shown in Figure 9.10, which is symmetric about p = 1/2. Comparing the 
curve in this figure with that in Figure 9.2, we may make the following observations: 


1. When the channel is noise free, permitting us to set p = 0, the channel capacity C attains 
its maximum value of one bit per channel use, which is exactly the information in each 
channel input. At this value of p, the entropy function H(p) attains its minimum value 
of zero. 

2. When the condirional probability of error p = 1/2 due to noise, the channel capacity C 
attains its minimum value of zero, whereas the entropy function H(p) attains its max- 
imum value of unity; in such a case the channel is said to be useless. 


| 9.8 Channel-Coding Theorem 


The inevitable presence of noise in a channel causes discrepancies (errors) between the 
output and input data sequences of a digital communication system. For a relatively noisy 
channel (e.g., wireless communication channel), the probability of error may reach a value 
as high as 1077, which means that (on the average) only 9 out of 10 transmitted bits are 
received correctly. For many applications, this level of reliability is unacceptable. Indeed, 
a probability of error equal to 107° or even lower is often a necessary requirement. To 
achieve such a high level of performance, we resort to the use of channel coding. 

The design goal of channel coding is to increase the resistance of a digital commu- 
nication system to channel noise. Specifically, channel coding consists of mapping the 
incoming data sequence into a channel input sequence, and inverse mapping the channel 
output sequence into an output data sequence in such a way that the overall effect of 
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channel noise on the system is minimized. The first mapping operation is performed in the 
transmitter by a channel encoder, whereas the inverse mapping operation is performed in 
the receiver by a channel decoder, as shown in the block diagram of Figure 9.11; to simpli 
the exposition, we have not included source encoding (before channel encoding) and source 
decoding (after channel decoding) in Figure 9.11. 

The channel encoder and channel decoder in Figure 9.11 are both under the de. 
signer’s control and should be designed to optimize the overall reliability of the commy. 
nication system. The approach taken is to introduce redundancy in the channel encoder 
so as to reconstruct the original source sequence as accurately as possible. Thus, in a rather 
loose sense, we may view channel coding as the dual of source coding in that the forme; 
introduces controlled redundancy to improve reliability, whereas the latter reduces redun. 
dancy to improve efficiency. 

The subject of channel coding is treated in detail in Chapter 10. For the purpose of 
our present discussion, it suffices to confine our attention to block codes. In this class of 
codes, the message sequence is subdivided into sequential blocks each k bits long, and each 
k-bit block is mapped into an n-bit block, where n > k. The number of redundant bits 
added by the encoder to each transmitted block is n — k bits. The ratio k/n is called the 
code rate. Using r to denote the code rate, we may thus write 


pane 
n 
where, of course, r is less than unity. For a prescribed k, the code rate r (and therefore the 
system’s coding efficiency) approaches zero as the block length n approaches infinity. 
The accurate reconstruction of the original source sequence at the destination re- 
quires that the average probability of symbol error be arbitrarily low. This raises the 
following important question: Does there exist a channel coding scheme such that the 
probability that a message bit will be in error is less than any positive number e (i.e., as 
small as we want it), and yet the channel coding scheme is efficient in that the code rate 
need not be too small? The answer to this fundamental question is an emphatic “yes.” 
Indeed, the answer to the question is provided by Shannon’s second theorem in terms of 
the channel capacity C, as described in what follows. Up until this point, time has not 
played an important role in our discussion of channel capacity. Suppose then the discrete 
memoryless source in Figure 9.11 has the source alphabet $ and entropy H(Y) bits per 
source symbol. We assume that the source emits symbols once every T, seconds. Hence, 
the average information rate of the source is H(¥)/T, bits per second. The decoder delivers 
decoded symbols to the destination from the source alphabet $ and at the same source 
rate of one symbol every T, seconds. The discrete memoryless channel has a channel ca- 
pacity equal to C bits per use of the channel. We assume that the channel is capable of 
being used once every T, seconds. Hence, the channel capacity per unit time is C/T, bits 
per second, which represents the maximum rate of information transfer over the channel. 
We are now ready to state Shannon’s second theorem, known as the channel coding 


theorem. 
Discrete Discrete i 
memoryless > Channel memoryless granne] m> Destination 
source channe! 


Transmitter Receiver 
































Noise 


FIGURE 9.11 Block diagram of digital communication system. 
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Specifically, the channel coding theorem? for a discrete memoryless channel is stated 
in two parts as follows. 


(i) Let a discrete memoryless source with an alphabet F have entropy H(Y) and produce 
symbols once every T, seconds. Let a discrete memoryless channel have capacity C 
and be used once every T, seconds. Then, if 

HAY) C .. 
= s 9.61 
T T (9.61) 
there exists a coding scheme for which the source output can be transmitted over the 
channel and be reconstructed with an arbitrarily small probability of error. The 
parameter C/T. is called the critical rate. When Equation (9.61) is satisfied with 
the equality sign, the system is said to be signaling at the critical rate. 


Conversely, if 


G 





HP) C 
T, Te 


it is not possible to transmit information over the channel and reconstruct it with an 
arbitrarily small probability of error. 


The channel coding theorem is the single most important result of information the- 
ory. The theorem specifies the channel capacity C as a fundamental limit on the rate at 
which the transmission of reliable error-free messages can take place over a discrete 
memoryless channel. However, it is important to note the following: 


» The channel coding theorem does not show us how to construct a good code. Rather, 
the theorem should be viewed as an existence proof in the sense that it tells us that 
if the condition of Equation (9.61) is satisfied, then good codes do exist. (Later in 
Chapter 10 we describe several good codes for discrete memoryless channels.) 

» The theorem does not have a precise result for the probability of symbol error after 
decoding the channel output. Rather, it tells us that the probability of symbol error 
tends to zero as the length of the code increases, again provided that the condition 
of Equation (9.61) is satisfied. 


Note also that power and bandwidth constraints were hidden in the discussion presented 
here. Nevertheless, these two system constraints do actually show up in the channel matrix 
P of the discrete memoryless channel. This observation is readily confirmed by linking the 
results of Example 9.5 on the binary symmetric channel with the noise analysis for the 
PCM receiver presented in Section 5.3. 


E APPLICATION OF THE CHANNEL CODING THEOREM 
To BINARY SYMMETRIC CHANNELS 


Consider a discrete memoryless source that emits equally likely binary symbols (0s and 
1s) once every T, seconds. With the source entropy equal to one bit per source symbol (see 
Example 9.1), the information rate of the source is (1/T,) bits per second. The source 
sequence is applied to a channel encoder with code rate r. The channel encoder produces 
a symbol once every T, seconds. Hence, the encoded symbol transmission rate is (1/T,) 
symbols per second. The channel encoder engages a binary symmetric channel once every 
T, seconds. Hence, the channel capacity per unit time is (C/T,) bits per second, where C 
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is determined by the prescribed channel transition probability p in accordance with Equa. 
tion (9.60). Accordingly, the channel coding theorem [part (i)] implies that if 


i-l 
TÍT (9.62) 


the probability of error can be made arbitrarily low by the use of a suitable channel 
encoding scheme. But the ratio T,/T, equals the code rate of the channel encoder: 

ra (9.63) 
Hence, we may restate the condition of Equation (9.62) simply as 


r=C (9.64) 


That is, for r = C, there exists a code (with code rate less than or equal to C) capable of 
achieving an arbitrarily low probability of error. 


> EXAMPLE 9.6 Repetition Code 


In this example, we present a graphical interpretation of the channel coding theorem. We also 
bring out a surprising aspect of the theorem by taking a look at a simple coding scheme. 
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FIGURE 9.12 Illustrating significance of the channel coding theorem. 
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TABLE 9.3 Average probability of error 
for repetition code 





Code Rate, r = I/n Average Probability of Error, P, 
1 107? 
3 3x 10-4 
3 10 
4 4x 107 
ł 10-8 
a 5 x 107-7 


Consider first a binary symmetric channel with transition probability p = 10~*. For this 
value of p, we find from Equation (9.60) that the channel capacity C = 0.9192. Hence, from 
the channel coding theorem, we may state that for any e > 0 and r = 0.9192, there exists a 
code of large enouigh length 7 and code rate r, and an appropriate decoding algorithm, such 
that when the coded bit stream is sent over the given channel, the average probability of 
channel decoding error is less than e. This result is depicted in Figure 9.12 for the limiting 
value e = 1078. 

To put the significance of this result in perspective, consider next a simple coding scheme 
that involves the use of a repetition code, in which each bit of the message is repeated several 
times. Let each bit (0 or 1) be repeated 7 times, where n = 2m + 1 is an odd integer. For 
example, for n = 3, we transmit 0 and 1 as 000 and 111, respectively. Intuitively, it would 
seem logical to use a majority rule for decoding, which operates as follows: If in a block of n 
received bits (representing one bit of the message), the number of Os exceeds the number of 
1s, the decoder decides in favor of a 0. Otherwise, it decides in favor of a 1. Hence, an error 
occurs when m + 1 or more bits out of n = 2m + 1 bits are received incorrectly. Because of the 
assumed symmetric nature of the channel, the average probability of error P, is independent of 
the a priori probabilities of 0 and 1. Accordingly, we find that P, is given by (see Problem 9.24) 


n 

P= È (r)i Lapis (9.65) 
i=m+1 t 

where p is the transition probability of the channel. 

Table 9.3 gives the average probability of error P, for a repetition code, which is cal- 
culated by using Equation (9.65) for different values of the code rate r. The values given here 
assume the use of a binary symmetric channel with transition probability p = 107°. The 
improvement in reliability displayed in Table 9.3 is achieved at the cost of decreasing code 
rate. The results of this table are also shown plotted as the curve labeled “repetition code” in 
Figure 9.12. This curve illustrates the exchange of code rate for message reliability, which is 
a characteristic of repetition codes. 

This example highlights the unexpected result presented to us by the channel coding 
theorem. The result is that it is not necessary to have the code rate r approach zero (as in 
the case of repetition codes) so as to achieve more and more reliable operation of the com- 
munication link. The theorem merely requires that the code rate be less than the channel 
capacity C. 4 


9.9 Differential Entropy and Mutual 
Information for Continuous Ensembles 


The sources and channels considered in our discussion of information-theoretic concepts 
thus far have involved ensembles of random variables that are discrete in amplitude. In 
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this section, we extend some of these concepts to continuous random variables and random 
vectors. The motivation for doing so is to pave the way for the description of another 
fundamental limit in information theory, which we take up in Section 9.10. 

Consider a continuous random variable X with the probability density function 
x(x). By analogy with the entropy of a discrete random variable, we introduce the fo]. 
lowing definition: 


i 1 
W(X) = r fxlx) g| | dx (9.66) 


We refer to h(X) as the differential entropy of X to distinguish it from the ordinary or 
absolute entropy. We do so in recognition of the fact that although /(X) is a useful math- 
ematical quantity to know, it is not in any sense a measure of the randomness of X, 
Nevertheless, we justify the use of Equation (9.66) in what follows. We begin by viewing 
the continuous random variable X as the limiting form of a discrete random variable that 
assumes the value x, = & Ax, where k = 0, +1, +2,..., and Ax approaches zero, By 
definition, the continuous random variable X assumes a value in the interval [x,, x, + Ax] 
with probability fx(x,) Ax. Hence, permitting Ax to approach zero, the ordinary entropy 
of the continuous random variable X may be written in the limit as follows: 


u 1 
HOC) = fi, È fle a oe a) 
= lim | D fxi) oga( z5) Ax — log, Ax >, fix(xu) ax 
Ax0 | komo FI x(xp) ka (9.67 
-67) 
5 i ; 7 
= i fx(x) e(z) dx — Jim log, Ax ie fx(x) dx 


= þh(X) — lim log, Ax 
Ax+0 


where, in the last line, we have made use of Equation (9.66) and the fact that the total 
area under the curve of the probability density function f(x) is unity. In the limit as 
Ax approaches zero, ~log, Ax approaches infinity. This means that the entropy of a con- 
tinuous random variable is infinitely large. Intuitively, we would expect this to be true, 
because a continuous random variable may assume a value anywhere in the interval 
(—%, ©) and the uncertainty associated with the variable is on the order of infinity. We 
avoid the problem associated with the term log, Ax by adopting /(X) as a differential 
entropy, with the term —log, Ax serving as reference. Moreover, since the information 
transmitted over a channel is actually the difference between two entropy terms that have 
a common reference, the information will be the same as the difference between the cor- 
responding differential entropy terms. We are therefore perfectly justified in using the term 
}({X), defined in Equation (9.66), as the differential entropy of the continuous random 
variable X. 

When we have a continuous random vector X consisting of # random variables X:, 
X2; - - - , Xn, we define the differential entropy of X as the n-fold integral 


hX) = iz Frxfx) os| | dx (9.68) 


where f(x) is the joint probability density function of X. 
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® EXAMPLE 9.7 Uniform Distribution 


Consider a random variable X uniformly distributed over the interval (0, a). The probability 
density function of X is 


Í o<x< 
sws fe nar 
0, otherwise 


Applying Equation (9.66) to this distribution, we get 


74 
hX) = [ 7 loala) dx 


(9.69) 
= log a 


Note that log a < 0 for a < 1. Thus this example shows that, unlike a discrete random variable, 
the differential entropy of a continuous random variable can be negative. <q 


P EXAMPLE 9.8 Gaussian Distribution 


Consider an arbitrary pair of random variables X and Y, whose probability density functions 
are respectively denoted by fy(x) and fx(x) where x is merely a dummy variable. Adapting 
the fundamental inequality of Equation (9.12) to the situation at hand, we may write? 





* Í: x(x) Z 
f fxi) on fi) dx <0 (9.70) 
or, equivalently, 
-7 frt) logs frla) de = —[ frix) loge Fal) dx (9.71) 


The quantity on the left-hand side of Equation (9.71) is the differential entropy of the random 
variable Y; hence, 


WY) <= - fi. file) logs fx(x) dx (9.72) 


Suppose now the random variables X and Y are described as follows: 


® The random variables X and Y have the same mean p and the same variance o°. 
» The random variable X is Gaussian distributed as shown by 


t _(x- př 
Fx(x) em To? ) (9.73) 


Hence, substituting Equation (9.73) into Equation (9.72), and changing the base of the log- 
arithm from 2 to e = 2.7183, we get 








20? 


We now recognize the following properties of the random variable Y (given that its mean is 
p and its variance is 07): 


b(Y) = —log, e if fun © = Hh lost VE} dx (9.74) 


| roaa 
| i- afra) de = o 
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We may therefore simplify Equation (9.74) as 


b(Y) = $ log,(2mea*) (9.75) 


The quantity on the right-hand side of Equation (9.75) is in fact the differential entropy of 
the Gaussian random variable X: 


A(X) = 5 loga(2me07) (9.76) 
Finally, combining Equations (9.75) and (9.76), we may write 


X: Gaussian random variable 


b(Y) = b(X), { (9.77) 


Y: another random variable 
where equality holds if, and only if, Y = X. 

We may now summarize the results of this important example as two entropic properties 
of a Gaussian random variable: 


1. For a finite variance a7, the Gaussian random variable has the largest differential en- 
tropy attainable by any random variable. 

2, The entropy of a Gaussian random variable X is uniquely determined by the variance 
of X (i.e., it is independent of the mean of X). 


Indeed, it is because of Property 1 that the Gaussian channel model is so widely used as a 
conservative model in the study of digital communication systems. 4 


& MUTUAL INFORMATION 


Consider next a pair of continuous random variables X and Y. By analogy with Equa- 
tion (9.47), we define the mutual information between the random variables X and Y as 
follows: 





1x Y=] [7 faxy) logs |4 xel J dx dy (9.78) 


where fx,y(x, y) is the joint probability density function of X and Y, and fx(x |y) is the 
conditional probability density function of X, given that Y = y. Also, by analogy with 


Equations (9.45), (9.50), (9.43), and (9.44) we find that the mutual information I(X; Y) 
has the following properties: 


1. X; Y) = KY; X) (9.79) 

2. (X; Y)=0 (9.80) 

3. (X; Y) = b(X) — b(X|Y) (9.81) 
= WY) — b(Y|X) i 


The parameter h(X) is the differential entropy of X; likewise for h(Y). The parameter 
b{X|Y) is the conditional differential entropy of X, given Y; it is defined by the double 
integral (see Equation (9.41)) 


idea hs 1 
hX] Y) = i: ‘a fuyvlx, y) loga =r] dx dy (9.82) 
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The parameter 4(Y|X) is the conditional differential entropy of Y, given X; it is defined 
in a manner similar to (X| Y). 








l 9.10 Information Capacity Theorem 


In this section, we use the idea of mutual information to formulate the information capacity 
theorem for band-limited, power-limited Gaussian channels. To be specific, consider a 
zero-mean stationary process X(t) that is band-limited to B hertz. Let X}, k = 1,2,..., 
K, denote the continuous random variables obtained by uniform sampling of the process 
X(t) at the Nyquist rate of 2B samples per second. These samples are transmitted in T 
seconds over a noisy channel, also band-limited to B hertz. Hence, the number of samples, 
K, is given by 


K = 2BT (9.83) 


We refer to X, as a sample of the transmitted signal. The channel output is perturbed 
by additive white Gaussian noise (AWGN) of zero mean and power spectral density 
N,/2. The noise is band-limited to B hertz. Let the continuous random variables Y}, 
k = 1,2,..., K denote samples of the received signal, as shown by 


Y,=X,+N, k=1,2,...,K (9.84) 
The noise sample N, is Gaussian with zero mean and variance given by 
o = NB (9.85) 


We assume that the samples Y,, k = 1, 2,..., K are statistically independent. 

A channel for which the noise and the received signal are as described in Equations 
(9.84) and (9.85) is called a discrete-time, memoryless Gaussian channel. It is modeled as 
in Figure 9.13. To make meaningful statements about the channel, however, we have to 
assign a cost to each channel input. Typically, the transmitter is power limited; it is there- 
fore reasonable to define the cost as 


E[X2} =P, k=1,2,...,K (9.86) 


where P is the average transmitted power. The power-limited Gaussian channel described 
herein is of not only theoretical but also practical importance in that it models many 
communication channels, including line-of-sight radio and satellite links. 

The information capacity of the channel is defined as the maximum of the mutual 
information between the channel input X, and the channel output Y, over all distributions 
on the input X, that satisfy the power constraint of Equation (9.86). Let I(X,; Y,) denote 


a a Y, 


Ny 


Ficure 9.13 Model of discrete-time, memoryless Gaussian channel. 


598 


CHAPTER 9 2 FUNDAMENTAL LIMITS IN INFORMATION THEORY 


the mutual information between X, and Y,. We may then define the information capacity 
of the channel as 
C = max{I(X;; Y,):E[XZ] = P) (9.87) 
Fue 

where the maximization is performed with respect to fx,(x), the probability density func- 
tion of X,. 

The mutual information I(X;; Y,) can be expressed in one of the two equivalent 
forms shown in Equation (9.81). For the purpose at hand, we use the second line of this 
equation and so write 


W(X p53 Ya) = b(Y,) — bal Xx) (9.88) 


Since X; and N, are independent random variables, and their sum equals Y,, as in Equa- 
tion (9.84), we find that the conditional differential entropy of Y,, given Xz, is equal to 
the differential entropy of N, (see Problem 9.28): 


hY: | Xe) = (Na) (9.89) 
Hence, we may rewrite Equation (9.88) as 
KXz; Ye) = B(Ye) — B(Nz) (9.90) 


Since h(N,) is independent of the distribution of X}, maximizing I(X,; Y4) in accor- 
dance with Equation (9.87) requires maximizing b(Y,), the differential entropy of sample 
Y, of the received signal. For b(Y,) to be maximum, Y, has to be a Gaussian random 
variable (see Example 9.8). That is, the samples of the received signal represent a noiselike 
process. Next, we observe that since N, is Gaussian by assumption, the sample X; of the 
transmitted signal must be Gaussian too. We may therefore state that the maximization 
specified in Equation (9.87) is attained by choosing the samples of the transmitted signal 
from a noiselike process of average power P. Correspondingly, we may reformulate Equa- 
tion (9.87) as 


C = I(X,; Y,):X, Gaussian, E[X§] = P (9.91) 


where the mutual information I(X,; Y,) is defined in accordance with Equation (9.90). 
For the evaluation of the information capacity C, we proceed in three stages: 


1. The variance of sample Y; of the received signal equals P + 07. Hence, the use of 
Equation (9.76) yields the differential entropy of Y, as 


ALY.) = 3 logs[2me(P + o°)] (9.92) 


2. The variance of the noise sample N; equals a”. Hence, the use of Equation (9.76) 
yields the differential entropy of N, as 


L(N,) = 4 log,(27e07) (9.93) 


3. Substituting Equations (9.92) and (9.93) into Equation (9.90) and recognizing the 
definition of information capacity given in Equation (9.91), we get the desired result: 


1 
C= 3 loes(1 + 2) bits per transmission (9.94) 
o 


With the channel used K times for the transmission of K samples of the process X(t) 
in T seconds, we find that the information capacity per unit time is (K/T) times the result 
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given in Equation (9.94). The number K equals 2BT, as in Equation (9.83). Accordingly, 
we may express the information capacity in the equivalent form: 


C=B loe,(1 + a) bits per second (9.95) 


NoB 
where we have used Equation (9.85) for the noise variance a”. 

Based on the formula of Equation (9.95), we may now state Shannon’s third (and 
most famous) theorem, the information capacity theorem," as follows: 


The information capacity of a continuous channel of bandwidth B hertz, perturbed by 
additive white Gaussian noise of power spectral density No/2 and limited in bandwidth 
to B, is given by 


P R 
C=B lors(1 + <a) bits per second 


where P is the average transmitted power. 


The information capacity theorem is one of the most remarkable results of infor- 
mation theory for, in a single formula, it highlights most vividly the interplay among three 
key system parameters: channel bandwidth, average transmitted power (or, equivalently, 
average received signal power), and noise power spectral density at the channel output. 
The dependence of information capacity C on channel bandwidth B is linear, whereas its 
dependence on signal-to-noise ratio P/NoB is logarithmic. Accordingly, it is easier to in- 
crease the information capacity of a communication channel by expanding its bandwidth 
than increasing the transmitted power for a prescribed noise variance. 

The theorem implies that, for given average transmitted power P and channel band- 
width B, we can transmit information at the rate of C bits per second, as defined in 
Equation (9.95), with arbitrarily small probability of error by employing sufficiently com- 
plex encoding systems. It is not possible to transmit at a rate higher than C bits per second 
by any encoding system without a definite probability of error. Hence, the channel capacity 
theorem defines the fundamental limit on the rate of error-free transmission for a power- 
limited, band-limited Gaussian channel. To approach this limit, however, the transmitted 
signal must have statistical properties approximating those of white Gaussian noise. 


@ SPHERE PACKING’ 


To provide a plausible argument supporting the information capacity theorem, suppose 
that we use an encoding scheme that yields K code words, one for each sample of the 
transmitted signal. Let n denote the length (i.e., the number of bits) of each code word. It 
is presumed that the coding scheme is designed to produce an acceptably low probability 
of symbol error. Furthermore, the code words satisfy the power constraint; that is, the 
average power contained in the transmission of each code word with x bits is nP, where 
P is the average power per bit. 

Suppose that any code word in the code is transmitted. The received vector of x bits 
is Gaussian distributed with mean equal to the transmitted code word and variance equal 
to na, where a? is the noise variance. With high probability, the received vector lies inside 
a sphere of radius Vna”, centered on the transmitted code word. This sphere is itself 
contained in a larger sphere of radius Vn(P + o7), where n(P + a°) is the average power 
of the received vector. 
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FIGURE 9.14 The sphere-packing problem. 


We may thus visualize the picture portrayed in Figure 9.14. With everything inside 
a small sphere of radius Vo? assigned to the code word on which it is centered, it is 
reasonable to say that when this particular code word is transmitted, the probability that 
the received vector will lie inside the correct “decoding” sphere is high. The key question 
is: How many decoding spheres can be packed inside the larger sphere of received vectors? 
In other words, how many code words can we in fact choose? To answer this question, 
we first recognize that the volume of an #-dimensional sphere of radius r may be written 
as A,r”, where A, is a scaling factor. We may therefore make the following statements: 


© The volume of the sphere of received vectors is A,,[”(P + o7)]"?. 
» The volume of the decoding sphere is A,(na?)””. 


Accordingly, it follows that the maximum number of nonintersecting decoding spheres 
that can be packed inside the sphere of possible received vectors is 


AnP + 07) _ (: Pua 
Anka? a? (9.96) 


| = Dir!2) logy(1+ Pio 


Taking the logarithm of this result to base 2, we readily see that the maximum number of 
bits per transmission for a low probability of error is indeed as defined previously in 
Equation (9.94). 


> ExaMPLE 9.9 Reconfiguration of Constellation for Reduced Power 


To illustrate the idea of sphere packing, consider the 64-QAM square constellation of Figure 
9.15a. The figure depicts two-dimensional nonintersecting decoding spheres centered on the 
message points in the constellation. In trying to pack the decoding spheres as tightly as possible 
while maintaining the same Euclidean distance between the message points as before, we 
obtain the alternative constellation shown in Figure 9.156. With a common Euclidean dis- 
tance between the message points, the two constellations of Figure 9.15 produce approxi- 
mately the same bit error rate, assuming the use of a high enough signal-to-noise ratio over 
an AWGN channel; see, for example, Equation (5.95). However, comparing these two con- 
stellations, we find that the sum of squared Euclidean distances from the message points to 
the origin in Figure 9.156 is smaller than that in Figure 9.15a. It follows therefore that the 
tightly packed constellation of Figure 9.156 has an advantage over the square constellation 
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FIGURE 9.15 (a) Square 64-QAM constellation. (b) The most tightly coupled alternative to that 
of part a. 


of Figure 9.15a: a smaller transmitted average signal energy per symbol for the same bit error 
rate on an AWGN channel. 


9.11 Implications of the Information 
Capacity Theorem 


Now that we have an intuitive feel for the information capacity theorem, we may go on 
to discuss its implications in the context of a Gaussian channel that is limited in both 
power and bandwidth. For the discussion to be useful, however, we need an ideal frame- 
work against which the performance of a practical communication system can be assessed. 
To this end, we introduce the notion of an ideal system defined as one that transmits data 
at a bit rate R, equal to the information capacity C. We may then express the average 
transmitted power as 


P=E,C (9.97) 
where E, is the transmitted energy per bit. Accordingly, the ideal system is defined by the 
equation 

E3 EC 
== lona(1 EA s) (9.98) 


Equivalently, we may define the signal energy-per-bit to noise power spectral density ratio 

E,/Npo in terms of the ratio C/B for the ideal system as 
Ey _ 20/8 -1 
No C/B 


A plot of bandwidth efficiency R,/B versus E,/Np is called the bandwidth-efficiency dia- 
gram. A generic form of this diagram is displayed in Figure 9.16, where the curve labeled 


(9.99) 
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“capacity boundary” corresponds to the ideal system for which R, = C. Based on Figure 
9.16, we can make the following observations: 


1. For infinite bandwidth, the ratio E/N, approaches the limiting value 


DE 
No/=  B+2\No (9.100) 
= log 2 = 0.693 


This value is called the Shannon limit for an AWGN channel, assuming a code rate 
of zero, Expressed in decibels, it equals —1.6 dB. The corresponding limiting value 
of the channel capacity is obtained by letting the channel bandwidth B in Equation 
(9.95) approach infinity; we thus find that 


C. = lim C 
aie (9.101) 
P 
= N, log, e 
where e is the base of the natural logarithm. 
2. The capacity boundary, defined by the curve for the critical bit rate R, = C, separates 
combinations of system parameters that have the potential for supporting error-free 
transmission (R, < C) from those for which error-free transmission is not possible 
(R, > C). The latter region is shown shaded in Figure 9.16. 
3. The diagram highlights potential trade-offs among E,/No, R,/B, and probability of 
symbol error P,. In particular, we may view movement of the operating point along 
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FIGURE 9.16 Bandwidth-efficiency diagram. 
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a horizontal line as trading P, versus E/N, for a fixed R,/B. On the other hand, we 
may view movement of the operating point along a vertical line as trading P, versus 
R,/B for a fixed E,/No. 


> EXAMPLE 9.10 M-ary PCM 


In this example, we look at an M-ary PCM system in light of the channel capacity theorem 
under the assumption that the system operates above the error threshold. That is, the average 
probability of error due to channel noise is negligible. 

We assume that the M-ary PCM system uses a code word consisting of 2 code elements, 
each having one of M possible discrete amplitude levels; hence the name “M-ary.” From 
Chapter 3 we recall that for a PCM system to operate above the error threshold, there must 
be provision for a noise margin that is sufficiently large to maintain a negligible error rate due 
to channel noise. This, in turn, means there must be a certain separation between these M 
discrete amplitude levels. Call this separation ko, where k is a constant and g? = NoB is the 
noise variance measured in a channel bandwidth B. The number of amplitude levels M is 
usually an integer power of 2. The average transmitted power will be least if the amplitude 
range is symmetrical about zero. Then the discrete amplitude levels, normalized with respect 
to the separation ka, will have the values +1/2, +3/2,..., = (M — 1)/2. We assume that 
these M different amplitude levels are equally likely. Accordingly, we find that the average 
transmitted power is given by 


APE M-11], 
Paa (G) ++ a Te 
[M -1 
= Boel =) 


Suppose that the M-ary PCM system described herein is used to transmit a message 
signal with its highest frequency component equal to W hertz. The signal is sampled at the 
Nyquist rate of 2W samples per second. We assume that the system uses a quantizer of the 
midrise type, with L equally likely representation levels. Hence, the probability of occurrence 
of any one of the L representation levels is 1/L. Correspondingly, the amount of information 
carried by a-single sample of the signal is log, L bits. With a maximum sampling rate of 2W 
samples per second, the maximum rate of information transmission of the PCM system, mea- 
sured in bits per second, is given by 





(9.102) 





Rs = 2W log, L bits per second (9.103) 


Since the PCM system uses a code word consisting of n code elements, each having one of M 
possible discrete amplitude values, we have M” different possible code words. For a unique 
encoding process, we require 


L = M” (9.104) 
Clearly, the rate of information transmission in the system is unaffected by the use of 


an encoding process. We may therefore eliminate L between Equations (9.103) and (9.104) 
to obtain 


R, = 2Wn log, M bits per second (9.105) 


Equation (9.102) defines the average transmitted power required to maintain an M-ary 
PCM system operating above the error threshold. Hence, solving this equation for the number 
of discrete amplitude levels, M, we get 





1/2 
ad ) (9.106) 


m= (14 ee 
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where a? = NoB is the variance of the channel noise measured in a bandwidth B. Therefore 
substituting Equation (9.106) into Equation (9.105), we obtain 3 


12P 
R, = Wn loes( 1 + aed (9.107) 
The channel bandwidth B required to transmit a rectangular pulse of duration 1/2” W (rep. 
resenting a code element in the code word) is given by (see Chapter 3) 


B= aW 


where x is a constant with a value lying between 1 and 2. Using the minimum possible value 
x = 1, we find that the channel bandwidth B = nW. We may thus rewrite Equation (9.107) 
as 


R,=B lona(1 + A (9.108) 
The ideal system is described by Shannon’s channel capacity theorem, given in Equation (9.95), 
Hence, comparing Equation (9.108) with Equation (9.95), we see that they are identical if 
the average transmitted power in the PCM system is increased by the factor k?/12, compared 
with the ideal system. Perhaps the most interesting point to note about Equation (9.108) is 
that the form of the equation is right: Power and bandwidth in a PCM system are exchanged 
on a logarithmic basis, and the information capacity C is proportional to the channel band- 
width B. 4 


> EXAMPLE 9.11 M-ary PSK and M-ary FSK 


In this example, we compare the bandwidth-power exchange capabilities of M-ary PSK and 
M-ary FSK signals in light of Shannon’s information capacity theorem. Consider first a co- 
herent M-ary PSK system that employs a nonorthogonal set of M phase-shifted signals for the 
transmission of binary data. Each signal in the set represents a symbol with log, M bits. Using 
the definition of null-to-null bandwidth, we may express the bandwidth efficiency of M-ary 
PSK as follows [see Equation (6.51)]: 


In Figure 9.174, we show the operating points for different numbers of phase levels M = 2, 
4, 8, 16, 32, 64. Each point corresponds to an average probability of symbol error P, = 107. 
In the figure we have also included the capacity boundary for the ideal system. We observe 
from Figure 9.17a that as M is increased, the bandwidth efficiency is improved, but the value 
of E,/Np required for error-free transmission moves away from the Shannon limit. 

Consider next a coherent M-ary FSK system that uses an orthogonal set of M frequency- 
shifted signals for the transmission of binary data, with the separation between adjacent signal 
frequencies set at 1/2T, where T is the symbol period. As with the M-ary PSK, each signal in 
the set represents a symbol with log, M bits. The bandwidth efficiency of M-ary FSK is as 
follows [see Equation (6.143)]: 


Ra _ 2 log, M 
B M 


In Figure 9.17b, we show the operating points for different numbers of frequency levels M = 
2, 4, 8, 16, 32, 64 for an average probability of symbol error P, = 1075. In the figure, we 
have also included the capacity boundary for the ideal system. We see that increasing M in 
(orthogonal) M-ary FSK has the opposite effect to that in (nonorthogonal) M-ary PSK. In 
particular, as M is increased, which is equivalent to increased bandwidth requirement, the 
operating point moves closer to the Shannon limit. 4 
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FIGURE 9.17 (a) Comparison of M-ary PSK against the ideal system for P, = 10°° and increas- 
ing M. (b) Comparison of M-ary FSK against the ideal system for P, = 107° and increasing M. 


® EXAMPLE 9.12 Capacity of Binary-Input AWGN Channel 


In this example, we investigate the capacity of an AWGN channel using encoded binary an- 
tipodal signaling (i.e., levels —1 and + 1 for binary symbols 0 and 1, respectively). In partic- 
ular, we address the issue of determining the minimum achievable bit error rate as a function 
of E,/No for varying code rate r. It is assumed that the binary symbols 0 and 1 are 
equiprobable. 

Let the random variables X and Y denote the channel input and channel output, re- 
spectively; X is a discrete variable, whereas Y is a continuous variable. In light of the second 
line of Equation (9.81), we may express the mutual information between the channel input 
and channel output as 


KX; Y) = b(¥) — WY |X) 


The second term, b(Y | X), is the conditional differential entropy of the channel output Y, given 
the channel input X. By virtue of Equations (9.89) and (9.93), this term is just the entropy of 
a Gaussian distribution. Hence, using o° to denote the variance of the channel noise, we may 
write 


h(Y|X) = Z loga(2 res?) 


Next, the first term, h(Y)}, is the differential entropy of the channel output Y. With the use of 
binary antipodal signaling, the probability density function of Y, given X = x, is a mixture 
of two Gaussian distributions with common variance g° and mean values —1 and +1, as 
shown by 


1 [pec + 1/207) | exp(=(y = oe ann 


fr(y;|%) “5 Vine Vino 
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Hence, we may determine the differential entropy of Y using the formula 


bY) = a frly:l x) logalfyly;|x)] dy; 


where fy(y;|x) is defined by Equation (9.109). From the formulas of h(Y |X) and ACY), it is 
clear that the mutual information is solely a function of the noise variance o7. Using Mio?) 
to denote this functional dependence, we may thus write 


KX; Y) = M(o°) 


Unfortunately, there is no closed formula that we can derive for M(o”) because of the difficulty 
of determining 4(Y). Nevertheless, the differential entropy b(Y) can be well approximated 
using Monte Carlo integration, which is straightforward to program on a digital computer; 
see Problem 9.36. 

Because symbols 0 and 1 are equiprobable, it follows that the channel capacity C is 
equal to the mutual information between X and Y. Hence, for error-free data transmission 
over the AWGN channel, the code rate r must satisfy the condition 


r< M(o?) (9.110) 
A robust measure of the ratio E,/No is 
Ey PP 
No Nor 20°r 


where P is the average transmitted power, and N,/2 is the two-sided power spectral density 
of the channel noise. Without loss of generality, we may set P = 1. We may then express the 
noise variance as 


No 





S (9.111) 

Substituting Equation (9.111) into (9.110) and rearranging terms, we get the desired relation: 
E, 1 

No 2rM~\{r) Qeta 





Minimum E,/No, dB 
= 
Minimum bit error rate 


-1 
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FiGuRE 9.18 Binary antipodal signaling over an AWGN channel. (a) Minimum E,/Np versus 
the code rate r. (b) Minimum bit error rate (BER) versus E,,/No for varying code rate r. 
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where M~*(r) is the inverse of the mutual information between the channel input and output, 
expressed as a function of the code rate r. 

Using the Monte Carlo method to estimate the differential entropy 4(Y) and therefore 
M~‘(r), the plots of Figure 9.18 are computed.’? Figure 9.184 plots the minimum E;/Np versus 
the code rate r for error-free communication. Figure 9.186 plots the minimum achievable bit 
error rate versus E;/Ny with the code rate r as a running parameter. From Figure 9.18 we 
may draw the following conclusions: 


> For uncoded binary signaling (i.e., r = 1), an infinite E,/No is required for error-free 
communication, which agrees with what we know about uncoded data transmission 
over an AWGN channel. 

> The minimum E;,/No decreases with decreasing code rate 7, which is intuitively satis- 
fying. For example, for r = 1/2, the minimum value of E,/No is slightly less than 0.2 
dB. 

» As approaches zero, the minimum E,/Ng approaches the limiting value of —1.6 dB, 
which agrees with the Shannon limit derived earlier; see Equation (9.100). <4 


9.12 Information Capacity of Colored 
Noise Channel"? 


The information capacity theorem as formulated in Equation (9.95) applies to a band- 
limited white noise channel. In this section, we extend Shannon’s information capacity 
theorem to the more general case of a nonwhite, or colored, noise channel. To be specific, 
consider the channel model shown in Figure 9.19a where the transfer function of the 
channel is denoted by H( f). The channel noise z(t), which appears additively at the channel 
output, is modeled as the sample function of a stationary Gaussian process of zero mean 
and power spectral density 5x(f). The requirement is twofold: 


1. Find the input ensemble, described by the power spectral density Sx(f), that maxi- 
mizes the mutual information between the channel output y(t) and the channel input 
x(t), subject to the constraint that the average power of x(t) is fixed at a constant 
value P. 

2. Hence, determine the optimum information capacity of the channel. 


This problem is a constrained optimization problem. To solve it, we proceed as follows: 


> Because the channel is linear, we may replace the model of Figure 9.194 with the 
equivalent model shown in Figure 9.19. From the viewpoint of the spectral char- 
acteristics of the signal plus noise measured at the channel output, the two models 
of Figure 9.19 are equivalent, provided that the power spectral density of the noise 
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FIGURE 9.19 (a) Model of band-limited, power-limited noisy channel. (b) Equivalent model of 
the channel. 
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n'(t) in Figure 9.19 is defined in terms of the power spectral density of the noise 
n(t) in Figure 9.194 as 


Snif) 
| Hif)|? 


where | H(f)| is the magnitude response of the channel. 

> To simplify the analysis, we use the “principle of divide and conquer” in a manner 
similar to that described in Section 6.12. Specifically, the channel is divided into a 
large number of adjoining frequency slots, as illustrated in Figure 9.20. The smaller 
we make the incremental frequency interval Af of each subchannel, the better is this 
approximation. 





Swf) = (9.113) 


The net result of these two points is that the original model of Figure 9.194 is replaced by 
the parallel combination of a finite number of subchannels, N, each of which is corrupted 
essentially by “band-limited white Gaussian noise.” 

The kth subchannel in the approximation to the model of Figure 9.196 is described 
by 


yelt) = xlt) + mt), k= 1,2,...,N (9,14) 
The average power of the signal component x, (£) is 
P, = Sx(f,) Af, k=1,2,...,N (9.115) 


where Sx(f,) is the power spectral density of the input signal evaluated at the frequency 
f = f,. The variance of the noise component 7, (ż) is 


Sx fe) 
|H(f,) |? 
where Sy(f,) and | H(f,) | are the noise spectral density and the channel’s magnitude re- 


sponse evaluated at the frequency fp, respectively. The information capacity of the kth 
subchannel is 


Af, k=1,2,...,N (9.116) 


oi = 





1 
C, = = Af loas(1 + 2), k=1,2,...,N (9.117) 
2 Cr, 
IPI Staircase 
E approximation 
Fa 
ANN Actual 
response 








f 


FIGURE 9.20 Staircase approximation of an arbitrary magnitude response | H(f) |; only positive- 
frequency portion of the response is shown. 
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where the factor 1/2 accounts for the fact that Af applies to both positive and negative 
frequencies. All the N subchannels are independent of one another. Hence the total ca- 
pacity of the overall channel is approximately given by the summation 

N 


C= Ca 

1 

x P 
> Af los (1 + 2) 
k=1 Tk 


The problem we have to address is to maximize the overall information capacity C subject 
to the constraint: 


(9.118) 


NI PR 7 


Mz 


P, = P = constant (9.119) 
k 


1 


The usual procedure to solve a constrained optimization problem is to use the method of 
Lagrange multipliers; see Note 19 in Chapter 6. To proceed with this optimization, we 
first define an objective function that incorporates both the information capacity C and 
the constraint [i.e., Equations (9.118) and (9.119)], as shown by 


J=} of oe, (1 + 2) + Alp -5 P.) (9.120) 
2 i Tk k=1 


where A is the Lagrange multiplier. Next, differentiating the objective function J with 
respect to P, and setting the result equal to zero, we obtain 


Af loge _ 


P, + o% at 


To satisfy this optimizing solution, we impose the following requirement: 


P,+ot=KAf fork =1,2,...,N (9.121) 


where K is a constant that is the same for all &. The constant K is chosen to satisfy the 
average power constraint. 

Inserting the defining values of Equations (9.115) and (9.116) in the optimizing con- 
dition of Equation (9.121), simplifying, and rearranging terms, we get 











Sx fe) 
s K , k=1,2,...,N 9.122 
x (fx) | Hf.) | ( ) 
Let F, denote the frequency range for which the constant K satisfies the condition 
Snl f) 
K= 
|H(f) |? 


Then, as the incremental frequency interval Af is allowed to approach zero and the number 
of subchannels N goes to infinity, we may use Equation (9.122) to formally state that the 
power spectral density of the input ensemble that achieves the optimum information ca- 
pacity is a nonnegative quantity defined by 


sulf) z 
stf=4% anp or teFa (9.123) 


0 otherwise 
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Since the average power of a random process is the total area under the curve of the power 
spectral density of the process, we may express the average power of the channel input 


x(t) as 
SN) 4 
Pees, (x an) 5 ee) 


For a prescribed P and specified Sn(f) and H(f), the constant K is the solution to Equation 
(9.124). 

The only thing that remains for us to do is to find the optimum information capacity, 
Substituting the optimizing solution of Equation (9.121) into Equation (9.118) and then 
using the defining values of Equations (9.115) and (9.116), we obtain 


N 
| HUF.) | `) 
maf ve:(x Snif.) 


When the incremental frequency interval Af is allowed to approach zero, this equation 
takes the limiting form; 


1 
cag 


1 T ( |H(f) P) 
C=> log.| K | d 9.125 
als ae a aes 
where the constant K is chosen as the solution to Equation (9.124) for a prescribed input 
signal power P. 


WATER-FILLING INTERPRETATION 
OF THE INFORMATION CAPACITY THEOREM 


Equations (9.123) and (9.124) suggest the picture portrayed in Figure 9.21. Specifically, 
we make the following observations: 


> The appropriate input power spectral density Sx(f) is described as the bottom regions 
of the function Sx(f)/| H(f) |? that lie below the constant level K, which are shown 
shaded. 


> The input power P is defined by the total area of these shaded regions. 


The spectral domain picture portrayed here is called the water-filling (pouring) inter- 
pretation in the sense that the process by which the input power is distributed across 


Swf) 
HAI? 

















FIGURE 9.21 Water-filling interpretation of information-capacity theorem for a colored noisy 
channel. 
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the function Sx(f)/|H(f) |? is identical to the way in which water distributes itself in a 
vessel. 

Consider now the idealized case of a band-limited signal in additive white Gaussian 
noise of power spectral density N(f) = No/2. The transfer function H(f) is that of an ideal 
band-pass filter defined by 


B B 
Hf) = 1, osf.-Z=lflsfetyz 


0, otherwise 


where f, is the midband frequency and B is the channel bandwidth. For this special case, 
Equations (9.124) and (9.125) reduce to, respectively, 


=- -Mo 
P = a(x o) 


2K 
C = B log,{ — 
(2) 


Hence, eliminating K between these two equations, we get the standard form of Shannon’s 
capacity theorem, defined by Equation (9.95). 


and 


® EXAMPLE 9.13 Capacity of NEXT-Dominated Channel 


From the discussion presented in Section 4.8, we recall that a major channel impairment in 
digital subscriber lines is near-end crosstalk (NEXT). The power spectral density of this cross- 
talk may be taken as 


Sx(f) = |Hxexrlf) |? Sx(f) (9.126) 


where Sx(f) is the power spectral density of the transmitted signal and Hyexr(f) is the transfer 
function that couples adjacent twisted pairs. The only constraint we have to satisfy in this 
example is that the power spectral density function Sx(f) be nonnegative for all f. Substituting 
Equation (9.126) into (9.123), we readily find that this condition is satisfied by solving for 
Kas 


| Hyexcr(f) | 
|H(f)|? 


Finally, using this result in Equation (9.125), we find that the capacity of the NEXT-dominated 
digital subscriber channel is given by 


K= (ı + Jun 


1 | Hf) |? 
org k lon(1 t Telf) >) ay 


where F4 is the set of positive and negative frequencies for which Sx(f) > 0. 


| 9.13 Rate Distortion Theory 


In Section 9.3 we introduced the source coding theorem for a discrete memoryless source, 
according to which the average code-word length must be at least as large as the source 
entropy for perfect coding (i-e., perfect representation of the source). However, in many 
practical situations there are constraints that force the coding to be imperfect, thereby 
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resulting in unavoidable distortion. For example, constraints imposed by a communication 
channel may place an upper limit on the permissible code rate and therefore average code- 
word length assigned to the information source. As another example, the information 
source may have a continuous amplitude as in the case of speech, and the requirement is 
to quantize the amplitude of each sample generated by the source to permit its represen- 
tation by a code word of finite length as in pulse-code modulation. In such cases, the 
problem is referred to as source coding with a fidelity criterion, and the branch of infor- 
mation theory that deals with it is called rate distortion theory.” Rate distortion theory 
finds applications in two types of situations: 


> Source coding where the permitted coding alphabet cannot exactly represent the 
information source, in which case we are forced to do lossy data compression. 


» Information transmission at a rate greater than channel capacity. 


Accordingly, rate distortion theory may be viewed as a natural extension of Shannon's 
coding theorems. 


RATE DISTORTION FUNCTION 


Consider a discrete memoryless source defined by an M-ary alphabet X: {x;|i = 1,2,..., 
M}, which consists of a set of statistically independent symbols together with the associated 
symbol probabilities {p;|i = 1, 2,..., M}. Let R be the average code rate in bits per code 
word. The representation code words are taken from another alphabet Y: {y;|7=1,2,..., 
N}. The source coding theorem states that this second alphabet provides a perfect repre- 
sentation of the source provided that R > H, where H is the source entropy. But if we are 
forced to have R < H, then there is unavoidable distortion and therefore loss of 
information. 

Let p(x; y;) denote the joint probability of occurrence of source symbol x; and rep- 
resentation symbol y;. From probability theory, we have 


pixa y) = Ply; xi) P(x) _ (9.127) 


where p(y;|x;) is a transition probability. Let d(x;, y;) denote a measure of the cost incurred 
in representing the source symbol x; by the symbol y;; the quantity d(x; y;) is referred to 
as a single-letter distortion measure. The statistical average of d(x;, y;) over all possible 
source symbols and representation symbols is given by 


M N 
d= X D pledoly,|x) des yi) (9,128) 
i=] j= 
Note that the average distortion d is a nonnegative continuous function of the transition 
probabilities p(y;|x;) that are determined by the source encoder-decoder pair. 
A conditional probability assignment p(y;|x;) is said to be D-admissible if and only 
if the average distortion d is less than or equal to some acceptable value D. The set of all 
D-admissible conditional probability assignments is denoted by 


Pp = {p(y;|x):d = D} (9.129) 


For each set of transition probabilities, we have a mutual information 


M N 5 $ 
UX; Y) = X, X, peedetyjled loe(” a) (9.130) 
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FIGURE 9.22 Summary of rate distortion theory. 


A rate distortion function R(D) is defined as the smallest coding rate possible for which 
the average distortion is guaranteed not to exceed D. Let Pp denote the set to which the 
conditional probability p(y;|x;) belongs for a prescribed D. Then, for a fixed D we write’ 


R(D) = min I(X;Y) (9.131) 
P(¥;|x)EPD 
subject to the constraint 
N 
S plyix) =1 fori=1,2,...,M (9.132) 
j=l 


The rate distortion function R(D) is measured in units of bits if the base-2 logarithm is 
used in Equation (9.130). Intuitively, we expect the distortion D to decrease as the rate 
distortion function R(D) is increased. We may say conversely that tolerating a large dis- 
tortion D permits the use of a smaller rate for coding and/or transmission of information. 

Figure 9.22 summarizes the main parameters of rate distortion theory. In particular, 
given the source symbols {x;,} and their probabilities {p;} and given a definition of the single- 
letter distortion measure d(x;, y,), the calculation of the rate distortion function R(D) 
involves finding the conditional probability assignment p(y;|x;) subject to certain con- 
straints imposed on p(y;|x;). This is a variational problem, the solution of which is un- 
fortunately not straightforward in general. 


b EXAMPLE 9.14 Gaussian Source 


Consider a discrete-time, memoryless Gaussian source with zero mean and variance g°. Let x 
denote the value of a sample generated by such a source. Let y denote a quantized version of 
x that permits a finite representation of it. The squared error distortion 

dix, y) = (x — y? 


provides a distortion measure that is widely used for continuous alphabets. The rate distortion 
function for the Gaussian source with squared error distortion, as described herein, is given 
by 


1 a 
= log {=}, 0=D=e 

R(D) = 42 a (5) aed (9.133) 
0, D> o 


In this case, we see that R(D) > œ as D > 0, and R(D) = 0 for D = æ. 4 
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FIGURE 9.23 Reverse water-filling picture for a set of parallel Gaussian processes. 


P EXAMPLE 9.15 Set of Parallel Gaussian Sources 


Consider next a set of N independent Gaussian random variables (X,}1, where X; has zero 
mean and variance ø}. Using the distortion measure 


N 
d= > (x; — 2° 
i=} 


and building on the result of Example 9.14, we may express the rate distortion function for 
the set of parallel Gaussian sources described here as 


N 
R(D) = > L ts( Z) (9.134) 
i=1 D; 
where D; is itself defined by 
À if À < o; 
D; = e a= o? (9,135) 


and the constant A is chosen so as to satisfy the condition 
N 
ŠD =D (9.136) 
i=1 


Equations (9.135) and (9.136) may be interpreted as a kind of “water-filling in reverse,” as 
illustrated in Figure 9.23. First, we choose a constant A and only the subset of random variables 
whose variances exceed the constant A. No bits are used to describe the remaining subset of 
random variables whose variances are less than the constant A. 4 


| 9.14 Data Compression 


Rate distortion theory naturally leads us to consider the idea of data compression that 
involves a purposeful or unavoidable reduction in the information content of data froma 
continuous or discrete source. Specifically, we may think of a data compressor, or signal 
compressor, as a device that supplies a code with the least number of symbols for the 
representation of the source output, subject to a permissible or acceptable distortion. The 
data compressor thus retains the essential information content of the source output by 
blurring fine details in a deliberate but controlled manner. Accordingly, data compression 
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is a lossy operation in the sense that the source entropy is reduced (i.e., information is 
lost), irrespective of the type of source being considered. 

In the case of a discrete source, the reason for using data compression is to encode 
the source output at a rate smaller than the source entropy. By so doing, the source coding 
theorem is violated, which means that exact reproduction of the original data is no longer 
possible. 

In the case of a continuous source, the entropy is infinite, and therefore a signal 
compression code must always be used to encode the source output at a finite rate. Con- 
sequently, it is impossible to digitally encode an analog signal with a finite number of bits 
without producing some distortion. This statement is in perfect accord with the idea of 
pulse-code modulation, which was studied in Chapter 3. There it was shown that quan- 
tization, which is basic to the analog-to-digital conversion process in pulse-code modula- 
tion, always introduces distortion (known as quantization noise) into the transmitted sig- 
nal. A quantizer may therefore be viewed as a signal compressor. 

The uniform and nonuniform quantizers considered in Chapter 3 are said to be scalar 
quantizers in the sense that they deal with samples of the analog signal (i.e., continuous 
source output) one at a time. Each sample is converted into a quantized value, with the 
conversion being independent from sample to sample. A scalar quantizer is a rather simple 
signal compressor, which makes it attractive for practical use. Yet it can provide a sur- 
prisingly good performance; this is especially so if nonuniform quantization is used. 

There is another class of quantizers known as vector quantizers that use blocks of 
consecutive samples of the source output to form vectors, each of which is treated as a 
single entity. The essential operation in a vector quantizer is the quantization of a random 
vector?® by encoding it as a binary code word. The vector is encoded by comparing it with 
a codebook consisting of a set of stored reference vectors known as code vectors or pat- 
terns, Each pattern in the codebook is used to represent input vectors that are identified 
by the encoder to be similar to the particular pattern, subject to the maximization of an 
appropriate fidelity criterion. The encoding process in a vector quantizer may thus be 
viewed as a pattern matching operation. 

Let N be the number of code vectors in the codebook, & be the dimension of each 
vector (i.e., the number of samples in each pattern), and r be the coded transmission rate 
in bits per sample. These three parameters are related as follows: 


_ log, N 


m (9.137) 


r 


Then, assuming that the size of the code book is sufficiently large, the signal-to-quanti- 
zation noise ratio (SNR) for the vector quantizer is given by 


ws x) + C, dB (9.138) 





10 logio(SNR) = e 


where C; is a constant (expressed in dB) that depends on the dimensions k. According to 
Equation (9.138), the SNR for a vector quantizer increases approximately at the rate of 
6/k dB for each doubling of the codebook size. Equivalently, we may state that the SNR 
increases by 6 dB per unit increase in rate (bits per sample) as in the standard PCM using 
a uniform scalar quantizer. The advantage of the vector quantizer over the scalar quantizer 
is that its constant term C, has a higher value, because the vector quantizer optimally 
exploits the correlations among the samples constituting a vector. Specifically, the constant 
C; increases with the dimension k, approaching the ultimate rate-distortion limit for a 
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given source of information. However, the improvement in SNR is attained at the cost of 
increased encoding complexity, which grows exponentially with the dimension k for a 
specified rate 7. Unfortunately, this is the main obstacle to the wide use of vector quanti- 
zation in practice. Nevertheless, in certain applications, the issue of computational com- 
plexity is mitigated by exploiting the capability of VLSI technology to concentrate a highly 
complex signal processor on a silicon chip. For example, that is precisely what is done in 
the use of code-excited linear predictive (CELP) modeling of speech in wireless commu- 
nication systems of the CDMA type, namely, the IS-95 system. From the description of 
CELP presented in Section 8.9, it is clear that the CELP modeling of speech is an example 
of vector quantization. 


| 9.15 Summary and Discussion 


In this chapter we established four fundamental limits on different aspects of a commu- 
nication system. The limits are embodied in the source coding theorem, the channel coding 
theorem, the information capacity theorem, and the rate distortion function. 

The source coding theorem, Shannon’s first theorem, provides the mathematical tool 
for assessing data compaction, that is, lossless compression of data generated by a discrete 
memoryless source. The theorem tells us that we can make the average number of binary 
code elements (bits) per source symbol as small as, but no smaller than, the entropy of the 
source measured in bits. The entropy of a source is a function of the probabilities of the 
source symbols that constitute the alphabet of the source. Since entropy is a measure of 
uncertainty, the entropy is maximum when the associated probability distribution gener- 
ates maximum uncertainty. 

The channel coding theorem, Shannon’s second theorem, is both the most surprising 
and the single most important result of information theory. For a binary symmetric chan- 
nel, the channel coding theorem tells us that for any code rate r less than or equal to the 
channel capacity C, codes do exist such that the average probability of error is as small as 
we want it. A binary symmetric channel is the simplest form of a discrete memoryless 
channel. It is symmetric because the probability of receiving a 1 if a 0 is sent is the same 
as the probability of receiving a 0 if a 1 is sent. This probability, the probability that an 
error will occur, is termed a transition probability. The transition probability p is deter- 
mined not only by the additive noise at the channel output but also by the kind of receiver 
used. The value of p uniquely defines the channel capacity C. 

Shannon’s third remarkable theorem, the information capacity theorem, tells us that 
there is a maximum to the rate at which any communication system can operate reliably 
(ie., free of errors) when the system is constrained in power. This maximum rate is called 
the information capacity, measured in bits per second. When the system operates at a rate 
greater than the information capacity, it is condemned to a high probability of error, 
regardless of the choice of signal set used for transmission or the receiver used for pro- 
cessing the received signal. 

Finally, the rate distortion function provides the mathematical tool for signal com- 
pression (i.c., solving the problem of source coding with a fidelity criterion): The rate 
distortion function can be applied to a discrete as well as continuous memoryless source. 

When the output of a source of information is compressed in a lossless manner, the 
resulting data stream usually contains redundant bits. These redundant bits can be removed 
by using a lossless algorithm such as Huffman coding or the Lempel—Ziv algorithm for 
data compaction. We may thus speak of data compression followed by data compaction 
as two constituents of the dissection of source coding, which is so called because it refers 
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exclusively to the sources of information. In some source coding applications, we have a 
third constituent, namely, data encryption, which follows data compaction. The purpose 
of data encryption is to disguise the data (bit) stream in such a way that it has no meaning 
to an unauthorized receiver. Some basic aspects of cryptography, which encompasses both 
encryption and decryption, follow quite naturally from information theory, as discussed 
in Appendix 5, Other issues relating to cryptography are also discussed in that appendix. 

One last comment is in order. Shannon’s information theory, as presented in this 
chapter, has been entirely in the context of memoryless sources and channels. The theory 
can be extended to deal with sources and channels with memory, in which case a symbol 
of interest depends on preceding symbols; however, the level of exposition needed to do 
this is beyond the scope of this book.” 


l NOTES AND REFERENCES 


1. According to Lucky (1989), the first mention of the term information theory by Shannon 
occurs in a 1945 memorandum entitled “A Mathematical Theory of Cryptography.” It is 
rather curious that the term was never used in the classic 1948 paper by Shannon, which 
laid down the foundations of information theory. For an introductory treatment of infor- 
mation theory, see Chapter 2 of Lucky (1989) and the paper by Wyner (1981); see also 
the books of Adámek (1991), Hamming (1980), and Abramson (1963). For more advanced 
treatments of the subject, see the books of Cover and Thomas (1991), Blahut (1987), 
McEliece (1977), and Gallager (1968). For a collection of papers on the development of 
information theory (including the 1948 classic paper by Shannon), see Slepian (1974). For 
a collection of the papers published by Shannon, see Sloane and Wyner (1993). 


2. The use of a logarithmic measure of information was first suggested by Hartley (1928); 
however, Hartley used logarithms to base 10. 


3. In statistical physics, the entropy of a physical system is defined by (Reif, 1967, p. 147) 
F = klog Q 


where k is Boltzmann’s constant, Q is the number of states accessible to the system, and 
log denotes the natural logarithm. This entropy has the dimensions of energy because its 
definition involves the constant &. In particular, it provides a quantitative measure of the 
degree of randomness of the system. Comparing the entropy of statistical physics with that 
of information theory, we see that they have a similar form. For a detailed discussion of 
the relation between them, see Pierce (1961, pp. 184-207) and Brillouin (1962). 


4. For the original proof of the source coding theorem, see Shannon (1948). A general proof 
of the source coding theorem is also given in the following books: Viterbi and Omura 
(1979, pp. 13-19), McEliece (1977, Chapter 3), and Gallager (1968, pp. 38-55). The 
source coding theorem is also referred to in the literature as the noiseless coding theorem, 
noiseless in the sense that it establishes the condition for error-free encoding to be possible. 


5. For proof of the Kraft-McMillan inequality, see Cover and Thomas (1991, pp. 82-84), 
Blahut (1990, pp. 298-299), and McEliece (1977, pp. 239-240). For a proof of Equation 
(9.23), see Cover and Thomas (1991), pp. 87-88), Blahut (1990, pp. 300-301), and 
McEliece (1977, pp. 241-242). 


6. The Huffman code is named after its inventor: D. A. Huffman (1952). For a readable 
account of Huffman coding and its use in data compaction, see Adamek (1991). 


7. The original papers on the Lempel—Ziv algorithm are Ziv and Lempel (1977, 1978). For 
readable descriptions of the Lempel—Ziv algorithm, see Lucky (1989, pp. 118-122), Blahut 
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(1990, pp. 314-319), and Gitlin, Hayes, and Weinstein (1992, pp. 120-122). For the 
application of the Lempel—Ziv algorithm to the compaction of English text, see Lucky 
(1989, pp. 122-128) and the paper by Welch (1984); see also the review paper by Weiss 
and Shremp (1993). 


. The channel coding theorem is also known as the noisy coding theorem. The original proof 


of the theorem is given in Shannon (1948). A proof of the theorem is also presented in 
Hamming (1980, Chapters 9 and 10) in sufficient detail so that a general appreciation of 
relevant results is developed. The second part of the theorem is referred to in the literature 
as the converse to the coding theorem. A proof of this theorem is presented in the following 
references: Viterbi and Omura (1979, pp. 28-34) and Gallager (1968, pp. 76-82). 


. The quantity 


T frix) io, 222) dx 


on the left-hand side of Equation (9.70) is called relative entropy or the Kullback-Leibler 
divergence between the probability density functions f(x) and f y(x); see Kullback (1968), 


Shannon’s information capacity theorem is also referred to in the literature as the Shannon- 
Hartley law in recognition of early work by Hartley on information transmission (Hartley, 
1928). In particular, Hartley showed that the amount of information that can be trans- 
mitted over a given channel is proportional to the product of the channel bandwidth and 
the time of operation. 


A lucid exposition of sphere packing is presented in Cover and Thomas (1991, pp. 242- 
243); see also Wozencraft and Jacobs (1965, pp. 323-341). 


Parts a and b of Figure 9.18 follow the corresponding parts of Figure 6.2 in the book by 
Frey (1998). 


For a rigorous treatment of the information capacity of a colored noisy channel, see Gal- 
lager (1968). The idea of replacing the channel model of Figure 9.194 with that of Figure 
9.198 is discussed in Gitlin, Hayes, and Weinstein (1992). 


For a complete treatment of rate distortion theory, see the book by Berger (1971); this 
subject is also treated in somewhat less detail in Cover and Thomas (1991), McEliece 
(1977), and Gallager (1968). 


For the derivation of Equation (9.131), see Cover and Thomas (1991, p. 345). An algorithm 
for computation of the rate distortion function R(D) defined in Equation (9.131) is de- 
scribed in Blahut (1987, pp. 220-221) and Cover and Thomas (1991, pp. 364-367). 


For the early papers on vector quantization, see Gersho (1979) and Linde, Buzo, and Gray 
(1980). For a tutorial review of vector quantization, see Gray (1984). Equation (9.138), 
defining the SNR for a vector quantizer, is discussed in Gersho and Cuperman (1985). For 
a complete treatment of vector quantization, see the book by Gersho and Gray (1992). 


For detailed discussion of discrete channels with memory, see Gallager (1968, pp. 97~112} 
and Ash (1965, pp. 211-229). 


Entropy 


9.1 Let p denote the probability of some event. Plot the amount of information gained by the 


occurrence of this event for 0 =p = 1. 
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9.2, A source emits one of four possible symbols during each signaling interval. The symbols 
occur with the probabilities: 


Po = 0.4 
pı = 0.3 
p: = 0.2 
p: = 0.1 


Find the amount of information gained by observing the source emitting each of these 
symbols. 

9.3 A source emits one of four symbols so, $1, $2, and sz with probabilities 1/3, 1/6, 1/4, and 
1/4, respectively. The successive symbols emitted by the source are statistically indepen- 
dent. Calculate the entropy of the source. 

9.4 Let X represent the outcome of a single roll of a fair die. What is the entropy of X? 

9.5 The sample function of a Gaussian process of zero mean and unit variance is uniformly 
sampled and then applied to a uniform quantizer having the input-output amplitude char- 
acteristic shown in Figure P9.5. Calculate the entropy of the quantizer output. 


Input 





FIGURE P9.5 


9.6 Consider a discrete memoryless source with source alphabet Y = {so, 51,..., Sx-1} and 
source statistics {po, P1, - - - , pr-1}. The nth extension of this source is another discrete 
memoryless source with source alphabet ¥” = {00, o1,..+ 5 Cm-1}, where M = K”. Let 
P(c) denote the probability of a;. 

(a) Show that 


x 


Pio) = 1 
0 


i 


which is to be expected. 


(b) Show that 


M-1 1 
È Pio) (+) =H), k=1,2,...,9% 
iso Pi, 
where p,, is the probability of symbol s;, and H(f) is the entropy of the original 
source. 


(c) Hence, show that 
M-1 


HS") = F Pla) logs p 


= nH(f) 
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9.7 Consider a discrete memoryless source with source alphabet 9 = {so, $1, 52} and source 
statistics {0.7, 0.15, 0.15}. 
(a) Calculate the entropy of the source. 
(b) Calculate the entropy of the second-order extension of the source. 


9.8 It may come as a surprise, but the number of bits needed to store text is much less than 
that required to store its spoken equivalent. Can you explain the reason for it? 


Data Compaction 


9.9 Consider a discrete memoryless source whose alphabet consists of K equiprobable 
symbols. 
-{a) Explain why the use of a fixed-length code for the representation of such a source is 
about as efficient as any code can be. 
(b) What conditions have to be satisfied by K and the code-word length for the coding 
efficiency to be 100 percent? 


9.10 Consider the four codes listed below: 





Symbol Code I Code IT Code III Code IV 
So 0 0 0 00 
Sy 10 01 01 01 
s 110 001 011 10 
S3 1110 0010 110 110 
S4 1111 0011 111 111 


(a) Two of these four codes are prefix codes. Identify them, and construct their individual 
decision trees. 
(b) Apply the Kraft-McMillan inequality to codes I, II, III, and IV. Discuss your results 
in light of those obtained in part (a). 
9.11 Consider a sequence of leters of the English alphabet with their probabilities of occurrence 
as given here: 


Letter a i I m n o p y 
Probability 0.1 0.1 0.2 0.1 0.1 0.2 0.1 0.1 


Compute two different Huffman codes for this alphabet. In one case, move a combined 
symbol in the coding procedure as high as possible, and in the second case, move it as 
low as possible. Hence, for each of the two codes, find the average code-word length and 
the variance of the average code-word length over the ensemble of letters. 

9.12 A discrete memoryless source has an alphabet of seven symbols whose probabilities of 
occurrence are as described here: 


Symbol So Sy So S3 S4 S5 S6 
Probability 0.25 0.25 0.125 0.125 0.125 0.0625 0.0625 


Compute the Huffman code for this source, moving a “combined” symbol as high as 
possible. Explain why the computed source code has an efficiency of 100 percent. 
9.13 Consider a discrete memoryless source with alphabet {so, s1, s2} and statistics {0.7, 0.15, 
0.15} for its output. 
(a) Apply the Huffman algorithm to this source. Hence, show that the average code- 
word length of the Huffman code equals 1.3 bits/symbol. 


Problems 621 


(b) Let the source be extended to order two. Apply the Huffman algorithm to the resulting 
extended source, and show that the average code-word length of the new code equals 
1.1975 bits/symbol. 

{c) Compare the average code-word length calculated in part (b) with the entropy of the 
original source. 

9,14 Figure P9.14 shows a Huffman tree. What is the code word for each of the symbols A, 
B, C, D, E, F, and G represented by this Huffman tree? What are their individual code- 
word lengths? 





FIGuRE P9.14 


9.15 A computer executes four instructions that are designated by the code words 
(00, 01, 10, 11). Assuming that the instructions are used independently with probabilities 
(1/2, 1/8, 1/8, 1/4), calculate the percentage by which the number of bits used for the 
instructions may be reduced by the use of an optimum source code. Construct a Huffman 
code to realize the reduction. 


9,16 Consider the following binary sequence 
11101001100010110100... 


Use the Lempel—Ziv algorithm to encode this sequence. Assume that the binary symbols 
0 and 1 are already in the codebook. 


Binary Symmetric Channel 


9.17 Consider the transition probability diagram of a binary symmetric channel shown in 
Figure 9.8. The input binary symbols 0 and 1 occur with equal probability. Find the 
probabilities of the binary symbols 0 and 1 appearing at the channel output. 

9.18 Repeat the calculation in Problem 9.17, assuming that the input binary symbols 0 and 1 
occur with probabilities 1/4 and 3/4, respectively. 


Mutual Information and Channel Capacity 


9.19 Consider a binary symmetric channel characterized by the transition probability p. Plot 
the mutual information of the channel as a function of p,, the a priori probability of 
symbol 1 at the channel input; do your calculations for the transition probability p = 0, 
0.1, 0.2, 0.3, 0.5. 
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9.20 Figure 9.10 depicts the variation of the channel capacity of a binary symmetric channel 
with the transition probability p. Use the results of Problem 9.19 to explain this variation 

9.21 Consider the binary symmetric channel described in Figure 9.8. Let po denote the prob. 
ability of sending binary symbol x9 = 0, and let p, = 1 — po denote the probability of 
sending binary symbol x, = 1. Let p denote the transition probability of the channel,” 


(a) Show that the mutual information between the channel input and channel output is 





given by 
KX; Y) = H(z) — H(p) 
where 
H(z) = z logs *) +(1-2) oe. 1 -) 
z = pop + (1 — po)(1 — p) 
and 





1 1 
Hip) = p ton(2) + (1 — p) e| = -) 


(b) Show that the value of po that maximizes I(%; Y) is equal to 1/2. 
(c) Hence, show that the channel capacity equals 


C= 1 - H(p) 


9.22 Two binary symmetric channels are connected in cascade, as shown in Figure P9.22. Find 
the overall channel capacity of the cascaded connection, assuming that both channels 
have the same transition probability diagram shown in Figure 9.8. 


Binary Binary 
Intput symmetric symmetric Output 
channel 1 channel 2 
FIGURE P9.22 


9.23 The binary erasure channel has two inputs and three outputs as described in Figure P9.23. 
The inputs are labeled 0 and 1, and the outputs are labeled 0, 1, and e. A fraction « of 
the incoming bits are erased by the channel. Find the capacity of the channel. 


o 
ao 


l-a 


FIGURE P9.23 
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9.24 Consider a digital communication system that uses a repetition code for the channel 
encoding/decoding. In particular, each transmission is repeated 7 times, where n = 2m + 
1 is an odd integer. The decoder operates as follows. If in a block of 7 received bits, the 
number of Os exceeds the number of 1s, the decoder decides in favor of a 0. Otherwise, 
it decides in favor of a 1. An error occurs when m + 1 or more transmissions out of n = 

¿2m + 1 are incorrect. Assume a binary symmetric channel. 


(a) For z = 3, show that the average probability of error is given by 
P. = 3p°(1 — p) + p° 
where p is the transition probability of the channel. 
(b) For 2 = 5, show that the average probability of error is given by 
P, = 10p*(1 — p) + Sp*(1 — p) + p° 
(c) Hence, for the general case, deduce that the average probability of error is given by 
n n . . 
P= D ("Jon py 
Differential Entropy 


9.25 Let Xi, X2,..., X, denote the elements of a Gaussian vector X. The X; are independent 
with mean y, and variance gẹ, i = 1, 2, . . . , n. Show that the differential entropy of the 
vector X equals 


A(X) = 5 logal2me(oio3 we 2) 


What does /(X) reduce to if the variances are equal? 
9.26 A continuous random variable X is constrained to a peak magnitude M; that is, 


-M<X<M. 
(a) Show that the differential entropy of X is maximum when it is uniformly distributed, 
as shown by 
Fala) 1/2M, -M<xsM 
x) = 
i 0, otherwise 


(b) Show that the maximum differential entropy of X is log, 2M. 
9.27 Prove the properties given in Equations (9.79) to (9.81) for the mutual information 
KX; Y). 
9.28 Consider the continuous random variable Y defined by 
Y=X+N 
where X and N are statistically independent. Show that the conditional differential en- 
tropy of Y, given X, equals 
h(Y|X) = hN) 
where A(N) is the differential entropy of N. 


Information Capacity 


9.29 A voice-grade channel of the telephone network has a bandwidth of 3.4 kHz. 
(a) Calculate the information capacity of the telephone channel for a signal-to-noise ratio 
of 30 dB. ; 
(b) Calculate the minimum signal-to-noise ratio required to support information trans- 
mission through the telephone channel at the rate of 9,600 b/s. 
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9.30 


9.31 


9.32 


9.33 


Alphanumeric data are entered into a computer from a remote terminal through a voice. 

grade telephone channel. The channel has a bandwidth of 3.4 kHz and output signal-tg. 

noise ratio of 20 dB. The terminal has a total of 128 symbols. Assume that the symbols 

are equiprobable and the successive transmissions are statistically independent. 

(a) Calculate the information capacity of the channel. 

(b) Calculate the maximum symbol rate for which error-free transmission over the chan. 

nel is possible. 

A black-and-white television picture may be viewed as consisting of approximately 

3 X 10° elements, each of which may occupy one of 10 distinct brightness levels with 

equal probability. Assume that (1) the rate of transmission is 30 picture frames per second 

and (2) the signal-to-noise ratio is 30 dB. ; 
Using the information capacity theorem, calculate the minimum bandwidth re. 

quired to support the transmission of the resulting video signal. 

Note: As a matter of interest, commercial television transmissions actually employ a band- 

width of 4.2 MHz, which fits into an allocated bandwidth of 6 MHz. 

In this problem, we continue with Example 9.9. Suppose that the tightly packed constel- 

lation of Figure 9.156 is scaled upward so that the transmitted signal energy per symbol 

is maintained at the same average value as that consumed by the 64-QAM square con- 

stellation of Figure 9.15a. Construct the new constellation that results from this scaling, 

How does the bit error rate of this new constellation compare with that of Figure 9,154? 

Justify your answer. 

The squared magnitude response of a twisted-pair channel can be modeled as 


| H(f)|? = exp(-aVf) 


The constant a is defined by 


where k is a constant depending on wire gauge, lo is a reference line length, and / is the 
actual length of the twisted pair under study. The squared magnitude response of the 
coupling responsible for NEXT has the form 


| Hyexc(f)|? = f°? 
where £ is a constant that depends on the type of cable used. 


Formulate the expression for the information capacity of the NEXT-dominated 
channel described here. 


Data Compression 


9.34 


Equation (9.138) for the signal-to-noise ratio (SNR) of a vector quantizer includes the 
SNR formula of Equation (3.33) for standard pulse-code modulation as a special case for 


” which k = 1. Justify the validity of this inclusion. 


9.35 


All practical data compression and data transmission schemes lie between two limits set 
by the rate distortion function and the channel capacity theorem. Both of these theorems 
involve the notion of mutual information, but in different ways. Elaborate on the issues 
raised by these two statements. 


Computer Experiment 


9.36 


In this problem, we revisit Example 9.12, which deals with coded binary antipodal sig- 
naling over an additive white Gaussian noise (AWGN) channel. Starting with Equation 
(9.112) and the underlying theory, develop a software package for computing the mini- 
mum E,/No required for a given bit error rate, where E; is the signal energy per bit, and 
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where p is the fraction of 
randomly chosen points that 
lie under the curve of g(y). 








FIGURE P9.36 


N,/2 is the noise spectral density. Hence, compute the results plotted in parts a and b of 
Figure 9.18. 

As mentioned in Example 9.12, the computation of the mutual information between 
the channel input and channel output is well approximated using Monte Carlo integra- 
tion. To explain how this method works, consider a function g(y) that is difficult to sample 
randomly, which is indeed the case for the problem at hand. (For our problem, the func- 
tion g(y) represents the complicated integrand in the formula for the differential entropy 
of the channel output.) For the computation, proceed as follows: 
> Find an area A that includes the region of interest and that is easily sampled. 

t Choose N points, uniformly randomly inside the area A. 

Then the Monte Carlo integration theorem states that the integral of the function g(y) 
with respect to y is approximately equal to the area A multiplied by the fraction of points 
that reside below the curve of g, as illustrated in Figure P9.36. The accuracy of the ap- 
proximation improves with increasing N. 





ERROR-CONTROL 
| CODING 


This chapter is the natural sequel to the preceding chapter on Shannon’s information 
theory. In particular, in this chapter we present error-control coding techniques that 
provide different ways of implementing Shannon’s channel-coding theorem. Each error- 
control coding technique involves the use of a channel encoder in the transmitter and a 
decoding algorithm in the receiver. 


The error-control coding techniques described herein include the following important 


classes of codes: 


P Linear block codes. 


> Cyclic codes. 


> Convolutional codes. 


> Compound codes exemplified by turbo codes and low-density parity-check codes, and 
their irregular variants. 


i 10.1 Introduction 
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The task facing the designer of a digital communication system is that of providing a cost- 
effective facility for transmitting information from one end of the system at a rate anda 
level of reliability and quality that are acceptable to a user at the other end. The two key 
system parameters available to the designer are transmitted signal power and channel 
bandwidth. These two parameters, together with the power spectral density of receiver 
noise, determine the signal energy per bit-to-noise power spectral density ratio E,/Np. In 
Chapter 6, we showed that this ratio uniquely determines the bit error rate for a particular 
modulation scheme. Practical considerations usually place a limit on the value that we can 
assign to E;,/No. Accordingly, in practice, we often arrive at a modulation scheme and find 
that it is not possible to provide acceptable data quality (i.e., low enough error perfor- 
mance). For a fixed E,/No, the only practical option available for changing data quality 
from problematic to acceptable is to use error-control coding. 

Another practical motivation for the use of coding is to reduce the required E,/No 
for a fixed bit error rate. This reduction in E,/No may, in turn, be exploited to reduce the 
required transmitted power or reduce the hardware costs by requiring a smaller antenna 
size in the case of radio communications. 

Error control" for data integrity may be exercised by means of forward error cor- 
rection (FEC). Figure 10.1a shows the model of a digital communication system using such 
an approach. The discrete source generates information in the form of binary symbols. 
The channel encoder in the transmitter accepts message bits and adds redundancy accord- 
ing to a prescribed rule, thereby producing encoded data at a higher bit rate. The channel 
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FIGURE 10.1 Simplified models of digital communication system. (a) Coding and modulation 
performed separately. (b) Coding and modulation combined. 


decoder in the receiver exploits the redundancy to decide which message bits were actually 
transmitted. The combined goal of the channel encoder and decoder is to minimize the 
effect of channel noise. That is, the number of errors between the channel encoder input 
(derivéd from the source) and the channel decoder output (delivered to the user) is 
minimized. 

For a fixed modulation scheme, the addition of redundancy in the coded messages 
implies the need for increased transmission bandwidth. Moreover, the use of error-control 
coding adds complexity to the system, especially for the implementation of decoding op- 
erations in the receiver. Thus, the design trade-offs in the use of error-control coding to 
achieve acceptable error performance include considerations of bandwidth and system 
complexity. 

There are many different error-correcting codes (with roots in diverse mathematical 
disciplines) that we can use. Historically, these codes have been classified into block codes 
and convolutional codes. The distinguishing feature for this particular classification is the 
presence or absence of memory in the encoders for the two codes. 

To generate an (n, k) block code, the channel encoder accepts information in suc- 
cessive k-bit blocks; for each block, it adds x — k redundant bits that are algebraically 
related to the k message bits, thereby producing an overall encoded block of bits, where 
n > k. The n-bit block is called a code word, and n is called the block length of the code. 
The channel encoder produces bits at the rate Ro = (#/k)R,, where R, is the bit rate of the 
information source. The dimensionless ratio r = k/n is called the code rate, where 
0 <r < 1. The bit rate Ro, coming out of the encoder, is called the channel data rate. 
Thus, the code rate is a dimensionless ratio, whereas the data rate produced by the source 
and the channel data rate are both measured in bits per second. 

In a convolutional code, the encoding operation may be viewed as the discrete- 
time convolution of the input sequence with the impulse response of the encoder. The 
duration of the impulse response equals the memory of the encoder. Accordingly, the 
encoder for a convolutional code operates on the incoming message sequence, using 
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a “sliding window” equal in duration to its own memory. This, in turn, means that jn 
a convolutional code, unlike a block code, the channel encoder accepts message bits as q 
continuous sequence and thereby generates a continuous sequence of encoded bits at 
a higher rate. 

In the model depicted in Figure 10.1a, the operations of channel coding and modu. 
lation are performed separately in the transmitter; likewise for the operations of detection 
and decoding in the receiver. When, however, bandwidth efficiency is of major concern, 
the most effective method of implementing forward error-control correction coding is + 
combine it with modulation as a single function, as shown in Figure 10.15. In such an 
approach, coding is redefined as a process of imposing certain patterns on the transmitted 
signal. 


AUTOMATIC-REPEAT REQUEST 


Feed-forward error correction (FEC) relies on the controlled use of redundancy in the 
transmitted code word for both the detection and correction of errors incurred during 
the course of transmission over a noisy channel. Irrespective of whether the decoding of 
the received code word is successful, no further processing is performed at the receiver, 
Accordingly, channel coding techniques suitable for FEC require only a one-way link be- 
tween the transmitter and receiver. 

There is another approach known as automatic-repeat request (ARQ)? for solving 
the error-control problem. The underlying philosophy of ARQ is quite different from that 
of FEC. Specifically, ARQ uses redundancy merely for the purpose of error detection. Upon 
the detection of an error in a transmitted code word, the receiver requests a repeat trans- 
mission of the corrupted code word, which necessitates the use of a return path (ie, a 
feedback channel). As such, ARQ can be used only on half-duplex or full-duplex links. In 
a half-duplex link, data transmission over the link can be made in either direction but not 
simultaneously. On the other hand, in a full-duplex link, it is possible for data transmission 
to proceed over the link in both directions simultaneously. 

A half-duplex link uses the simplest ARQ scheme known as the stop-and-wait strat- 
egy. In this approach, a block of message bits is encoded into a code word and transmitted 
over the channel, The transmitter then stops and waits for feedback from the receiver. The 
feedback signal can be acknowledgment of a correct receipt of the code word or a request 
for transmission of the code word because of an error in its decoding. In the latter case, 
the transmitter resends the code word in question before moving onto the next block of 
message bits. l 

The idling problem in stop-and-wait ARQ results in reduced data throughput, which 
is alleviated in another type of ARQ known as continuous ARQ with pullback. This second 
strategy uses a full-duplex link, thereby permitting the receiver to send a feedback signal 
while the transmitter is engaged in sending code words over the forward channel. Specif- 
ically, the transmitter continues to send a succession of code words until it receives a 
request from the receiver (on the feedback channel) for a retransmission. At that point, 
the transmitter stops, pulls back to the particular code word that was not decoded correctly 
by the receiver, and retransmits the complete sequence of code words starting with the 
corrupted one. 

Ina refined version of continuous ARQ known as the continuous ARQ with selective 
repeat, data throughout is improved further by only retransmitting the code word that. 
was received with detected errors. In other words, the need for retransmitting the success- 
fully received code words following the corrupted code word is eliminated. 
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The three types of ARQ described here offer trade-offs of their own between the 
need for a half-duplex or full-duplex link and the requirement for efficient use of com- 
munication resources. In any event, they all rely on two premises: 


> Error detection, which makes the design of the decoder relatively simple. 


> Noiseless feedback channel, which is not a severe restriction because the rate of 
information flow over the feedback channel is typically quite low. 


For these reasons, ARQ is widely used in computer-communication systems. 

Nevertheless, the fact that FEC requires only one-way links for its operation makes 
the FEC much wider in application than ARQ. Moreover, the increased decoding com- 
plexity of FEC due to the combined need for error detection and correction is no longer 
a pressing practical issue because the decoder usually lends itself to microprocessor or 
VLSI implementation in a cost-effective manner. 


i 10.2 Discrete-Memoryless Channels 


Returning to the model of Figure 10.12, the waveform channel is said to be memoryless 
if the detector output in a given interval depends only on the signal transmitted in that 
interval, and not on any previous transmission. Under this condition, we may model the 
combination of the modulator, the waveform channel, and the detector as a discrete 
memoryless channel. Such a channel is completely described by the set of transition prob- 
abilities p(/|7), where i denotes a modulator input symbol, j denotes a demodulator output 
symbol, and p(j| i) denotes the probability of receiving symbol j, given that symbol i was 
sent. (Discrete memoryless channels were described previously at some length in Section 
9.5.) 

The simplest discrete memoryless channel results from the use of binary input and 
binary output symbols, When binary coding is used, the modulator has only the binary 
symbols 0 and 1 as inputs. Likewise, the decoder has only binary inputs if binary quan- 
tization of the demodulator output is used, that is, a bard decision is made on the demod- 
ulator output as to which symbol was actually transmitted. In this situation, we have a 
binary symmetric channel (BSC) with a transition probability diagram as shown in Figure 
10.2. The binary symmetric channel, assuming a channel noise modeled as additive white 
Gaussian noise (AWGN) channel, is completely described by the transition probability p. 
The majority of coded digital communication systems employ binary coding with hard- 
decision decoding, due to the simplicity of implementation offered by such an approach. 
Hard-decision decoders, or algebraic decoders, take advantage of the special algebraic 
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FIGURE 10.2 Transition probability diagram of binary symmetric channel. 
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FIGURE 10.3 Binary input Q-aryʻoutput discrete memoryless channel. (a) Receiver for binary 
phase-shift keying. (b) Transfer characteristic of multilevel quantizer. (c) Channel transition prob- 
ability diagram. Parts (b) and (c) are illustrated for eight levels of quantization. 


structure that is built into the design of channe! codes to make the decoding relatively easy 
to perform. 

The use of hard decisions prior to decoding causes an irreversible loss of information 
in the receiver. To reduce this loss, soft-decision coding is used. This is achieved by in- 
cluding a multilevel quantizer at the demodulator output, as illustrated in Figure 10.34 
for the case of binary PSK signals. The input-output characteristic of the quantizer is 
shown in Figure 10.3b. The modulator has only the binary symbols 0 and 1 as inputs, but 
the demodulator output now has an alphabet with Q symbols. Assuming the use of the 
quantizer as described in Figure 10.3b, we have Q = 8. Such a channel is called.a binary 
input Q-ary output discrete memoryless channel. The corresponding channel transition 
probability diagram is shown in Figure 10.3c. The form of this distribution, and conse- 
quently the decoder performance, depends on the location of the representation levels of 
the quantizer, which, in turn, depends on the signal level and noise variance. Accordingly, 
the demodulator must incorporate automatic gain control if an effective multilevel quan- 
tizer is to be realized. Moreover, the use of soft decisions complicates the implementation 
of the decoder. Nevertheless, soft-decision decoding offers significant improvement in per- 
formance over hard-decision decoding by taking a probabilistic rather than an algebraic 
approach. It is for this reason that soft-decision decoders are also referred to as probabi- 
listic decoders. 


& CHANNEL CODING THEOREM REVISITED 





In Chapter 9, we established the concept of channel capacity, which, for a discrete 
memoryless channel, represents the maximum amount of information transmitted per 
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channel use. The channel coding theorem states that if a discrete memoryless channel has 
capacity C and a source generates information at a rate less than C, then there exists a 
coding technique such that the output of the source may be transmitted over the channel 
with an arbitrarily low probability of symbol error. For the special case of a binary sym- 
metric channel, the theorem tells us that if the code rate r is less than the channel capacity 
C, then it is possible to find a code that achieves error-free transmission over the channel. 
Conversely, it is not possible to find such a code if the code rate r is greater than the 
channel capacity C. 

The channel coding theorem thus specifies the channel capacity C as a fundamental 
limit on the rate at which the transmission of reliable (error-free) messages can take place 
over a discrete memoryless channel. The issue that matters is not the signal-to-noise ratio, 
so long as it is large enough, but how the channel input is encoded. 

The most unsatisfactory feature of the channel coding theorem, however, is its non- 
constructive nature. The theorem asserts the existence of good codes but does not tell us 
how to find them. By good codes we mean families of channel codes that are capable of 
providing reliable transmission of information (i.e., at arbitrarily small probability of sym- 
bol error) over a noisy channel of interest at bit rates up to a maximum value less than 
the capacity of that channel. The error-control coding techniques described in this chapter 
provide different methods of designing good codes, 


B NOTATION 


The codes described in this chapter are binary codes, for which the alphabet consists only 
of symbols 0 and 1. In such a code, the encoding and decoding functions involve the binary 
arithmetic operations of modulo-2 addition and multiplication performed on code words 
in the code. 

Throughout this chapter, we use an ordinary plus sign (+) to denote modulo-2 ad- 
dition. The use of this terminology will not lead to confusion because the whole chapter 
relies on binary arithmetic. In so doing, we avoid the use of a special symbol @, as we did 
in preceding chapters. Thus, according to the notation used in this chapter, the rules for 
modulo-2 addition are as follows: 


0+0=0 
1+0=1 
0+1=1 
1+1=0 
Because 1 + 1 = 0, it follows that 1 = —1. Hence, in binary arithmetic, subtraction is the 
same as addition, The rules for modulo-2 multiplication are as follows: 
0x0=0 
1x0=0 
0x1=0 
1x1=1 
Division is trivial in that we have 
1+1=1 


0+1=0 
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and division by 0 is not permitted. Modulo-2 addition is the EXCLUSIVE-OR operation 
in logic, and modulo-2 multiplication is the AND operation. 


a 10.3 Linear Block Codes 


A code is said to be linear if any two code words in the code can be added in modulo-2 
arithmetic to produce a third code word in the code. Consider then an (n, k) linear block 
code, in which k bits of the 2 code bits are always identical to the message sequence to be 
transmitted. The n — k bits in the remaining portion are computed from the message bits 
in accordance with a prescribed encoding rule that determines the mathematical structure 
of the code. Accordingly, these n — & bits are referred to as generalized parity check bits 
or simply parity bits. Block codes in which the message bits are transmitted in unaltered 
form are called systematic codes. For applications requiring both error detection and error 
correction, the use of systematic block codes simplifies implementation of the decoder. 

Let mo, Mi, . - . , Mg-1 constitute a block of k arbitrary message bits. Thus we have 
2* distinct message blocks. Let this sequence of message bits be applied to a linear block 
encoder, producing an n-bit code word whose elements are denoted by Co, Cis © © © 5 Crot 
Let bo, bi, ..., 5,—-¢-1 denote the (n — k) parity bits in the code word. For the code to 
possess a systematic structure, a code word is divided into two parts, one of which is 
occupied by the message bits and the other by the parity bits. Clearly, we have the option 
of sending the message bits of a‘code word before the parity bits, or vice versa. The former 
option is illustrated in Figure 10.4, and its use is assumed in the sequel. 

According to the representation of Figure 10.4, the (x — k) left-most bits of a code 
word are identical to the corresponding parity bits, and the k right-most bits of the code 


word are identical to the corresponding message bits. We may therefore write 
bi, - ¢=0,1,...,2-k-1 
G= . 
i Mik- t=n—kn-—k+1,...,n-1 


(10.1) 


The (n — k) parity bits are linear sums of the k message bits, as shown by the generalized 
relation 


bi = pomo + pum +++- + pr-i;Mk-1 (10.2) 
where the coefficients are defined as follows: 
= is if b; depends on 7n; 
i 


0 otherwise 


(10.3) 


The coefficients p; are chosen in such a way that the rows of the generator matrix are 
linearly independent and the parity equations are unique. 

The system of Equations (10.1) and (10.2) defines the mathematical structure of the 
(n, k) linear block code. This system of equations may be rewritten in a compact form 








| bo bi, +++ bpak | Mo M, + = +, M1 





s 


Parity bits Message bits 





FIGURE 10.4 Structure of systematic code word. 
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using matrix notation. To proceed with this reformulation, we define the 1-by-k message 
vector, or information vector, m, the 1-by-(n — k) parity vector b, and the 1-by-n code 
vector c as follows: 


m= Pno, Myers m1] (10.4) 
b= [bos bı, Seas bn-2-1] (10.5) 
c = [cos C1y ++ + 5 Ena]. (10.6) 


Note that all three vectors are row vectors. The use of row vectors is adopted in this chapter 
for the sake of being consistent with the notation commonly used in the coding literature. 
We may thus rewrite the set of simultaneous equations defining the parity bits in the 
compact matrix form: 


b = mP (10.7) 
where P is the k-by-(n — k) coefficient matrix defined by 


Poo Por >et Pop-k-1 


Pio Putt Pty kd 


P= (10.8) 


Pe-10 Pe-aya °° Pk-in-k-1 


where py is 0 or 1. 
From the definitions given in Equations (10.4)-(10.6), we see that c may be expressed 
as a partitioned row vector in terms of the vectors m and b as follows: 


c= [bim] (10.9) 


Hence, substituting Equation (10.7) into Equation (10.9) and factoring out the common 
message vector m, we get 


c= m[P: I] (10.10) 
where I, is the k-by-k identity matrix: 
l 1 0 
R=]. 2 : (10.11) 
0 0 1 
Define the k-by-n generator matrix 
G = P:i] (10.12) 


The generator matrix G of Equation (10.12) is said to be in the canonical form in that its 
k rows are linearly independent; that is, it is not possible to express any row of the matrix 
G as a linear combination of the remaining rows. Using the definition of the generator 
matrix G, we may simplify Equation (10.10) as 


c=mG (10.13) 


The full set of code words, referred to simply as the code, is generated in accordance 
with Equation (10.13) by letting the message vector m range through the set of all 24 
binary k-tuples (1-by-k vectors). Moreover, the sum of any two code words is another 
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code word. This basic property of linear block codes is called closure. To prove its validity, 
consider a pair of code vectors c; and c; corresponding to a pair of message vectors m; and 
m,, respectively. Using Equation (10.13) we may express the sum of c; and c; as 


G + c = m;G + mG 
= (m; + m)G 

The modulo-2 sum of m; and m; represents a new message Vector. Correspondingly, the 
modulo-2 sum of c; and c; represents a new code vector. 


There is another way of expressing the relationship between the message bits and 
parity-check bits of a linear block code. Let H denote an (n — k)-by-# matrix, defined as 


H = [i:i PY (10.14) 
where P7 is an (7 — k)-by-k matrix, representing the transpose of the coefficient matrix P, 


and I,_, is the (2 — k)-by-(# — k) identity matrix. Accordingly, we may perform the 
following multiplication of partitioned matrices: 


pe 
HG? = [in : Pl | 
I, 
= pT + pF 

where we have used the fact that multiplication of a rectangular matrix by an identity 
matrix of compatible dimensions leaves the matrix unchanged. In modulo-2 arithmetic, 
we have PT + PT = 0, where 0 denotes an (7 — k)-by-k null matrix (i.e., a matrix that has 
zeros for all of its elements). Hence, 


HG? = 0 (10.15) 


Equivalently, we have GH = 0, where 0 is a new null matrix. Postmultiplying both sides 
of Equation (10.13) by H7, the transpose of H, and then using Equation (10.15), we get 


cH? = mGH? 
=0 
The matrix H is called the parity-check matrix of the code, and the set of equations spec- 
ified by Equation (10.16) are called parity-check equations. 
The generator equation (10.13) and the parity-check detector equation (10.16) are 


basic to the description and operation of a linear block code. These two equations are 
depicted in the form of block diagrams in Figure 10.54 and 10.58, respectively. 


(10.16) 


Message vector m Code vector 









Parity-check 
matrix 
H 


Code vector Nuil vector 








(6) 


Ficure 10.5 Block diagram representations of the generator equation (10.13) and the parity- 
check equation (10.16). 
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& EXAMPLE 10.1 Repetition Codes 


Repetition codes represent the simplest type of linear block codes. In particular, a single mes- 
sage bit is encoded into a block of x identical bits, producing an (n, 1) block code. Such a 
code allows provision for a variable amount of redundancy. There are only two code words 
in the code: an all-zero code word and an all-one code word. 

Consider, for example, the case of a repetition code with k = 1 and 7 = 5. In this case, 
we have four parity bits that are the same as the message bit. Hence, the identity matrix I, = 
1, and the coefficient matrix P consists of a 1-by-4 vector that has 1 for all of its elements. 
Correspondingly, the generator matrix equals a row vector of all 1s, as shown by 


G=[1 1 1.1:4 


The transpose of the coefficient matrix P, namely, matrix P7, consists of a 4-by-1 vector that 
has 1 for all of its elements. The identity matrix I,_, consists of a 4-by-4 matrix. Hence, the 
parity-check matrix equals 


100 0:1 
010 0:1 
H= ; 
001 0:1 
000 1:1 


Since the message vector consists of a single binary symbol, 0 or 1, it follows from Equation 
(10.13) that there are only two code words: 00000 and 11111 in the (5, 1) repetition code, 
as expected. Note also that HGT = 0, modulo-2, in accordance with Equation (10.15), <4 


E SYNDROME: DEFINITION AND PROPERTIES 


The generator matrix G is used in the encoding operation at the transmitter. On the other 
hand, the parity-check matrix H is used in the decoding operation at the receiver. In the 
context of the latter operation, let r denote the 1-by-n received vector that results from 
sending the code vector c over a noisy channel. We express the vector r as the sum of the 
original code vector c and a vector e, as shown by 


r=cte (10.17) 


The vector e is called the error vector or error pattern. The ith element of e equals 0 if the 
corresponding element of r is the same as that of c. On the other hand, the ith element of 
e equals 1 if the corresponding element of r is different from that of c, in which case an 
error is said to have occurred in the ith location. That is, for i = 1, 2,..., 7, we have 


gods f if an error has occurred in the ith location (10.18) 


0 otherwise 


The receiver has the task of decoding the code vector c from the received vector r. 
The algorithm commonly used to perform this decoding operation starts with the com- 
putation of a 1-by-(a — k) vector called the error-syndrome vector or simply the syn- 
drome.’ The importance of the syndrome lies in the fact that it depends only upon the 
error pattern. 

Given a 1-by-n received vector r, the corresponding syndrome is formally defined as 


s = rH? (10.19) 


Accordingly, the syndrome has the following important properties. 
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Property 1 
The syndrome depends only on the error pattern, and not on the transmitted code word, 


To prove this property, we first use Equations (10.17) and (10.19), and then Equation 
(10.16) to obtain 
s = (c + e)H7 
cH? + eH? (10.20) 
= eH” 
Hence, the parity-check matrix H of a code permits us to compute the syndrome s, which 
depends only upon the error pattern e. Á 


Property 2 
All error patterns that differ by a code word have the same syndrome. 


For k message bits, there are 2* distinct code vectors denoted as c; i = 0, 1,..., 
2* — 1. Correspondingly, for any error pattern e, we define the 2* distinct vectors e; as 


e=e+c, i=0,1,...,2*-1 (10.21) 


The set of vectors {e;, i = 0, 1,..., 2* — 1) so defined is called a coset of the code. In 
other words, a coset has exactly 2* elements that differ at most by a code vector. Thus, 
an (n, k) linear block code has 2”~* possible cosets. In any event, multiplying both sides 
of Equation (10.21) by the matrix H”, we get 


e;H? = eH? + cH? 
= eH? 


which is independent of the index i. Accordingly, we may state that each coset of the code 
is characterized by a unique syndrome. 

We may put Properties 1 and 2 in perspective by expanding Equation (10.20). Spe- 
cifically, with the matrix H having the systematic form given in Equation (10.14), where 
the matrix P is itself defined by Equation (10.8), we find from Equation (10.20) that the 
(n — k) elements of the syndrome s are linear combinations of the n elements of the error 
pattern e, as shown by 


(10.22) 


So = Eo + Cn—RPoo + Cn-k+iPio T °° * + Cn—-1Pe-10 
S1 = ey + Cy—ePor + Enriba ttt + Cn-1Pk-1,1 (10.23) 
Sn-k-3 = En-k-1 + En—kPop-k-1 Ft E Cn-1Pk-1,n-k-1 


This set of (7 — k) linear equations clearly shows that the syndrome contains information 
about the error pattern and may therefore be used for error detection. However, it should 
be noted that the set of equations is underdetermined in that we have more unknowns 
than equations. Accordingly, there is xo unique solution for the error pattern. Rather, 
there are 2” error patterns that satisfy Equation (10.23) and therefore result in the same 
syndrome, in accordance with Property 2 and Equation (10.22). In particular, with 27 
possible syndrome vectors, the information contained in the syndrome s about the error 
pattern e is not enough for the decoder to compute the exact value of the transmitted code 
vector. Nevertheless, knowledge of the syndrome s reduces the search for the true error 
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pattern e from 2” to 2"~* possibilities. Given these possibilities, the decoder has the task 
of making the best selection from the cosets corresponding to s. 


a MINIMUM DISTANCE CONSIDERATIONS 


Consider a pair of code vectors c, and c, that have the same number of elements. The 
Hamming distance d(c,, c2) between such a pair of code vectors is defined as the number 
of locations in which their respective elements differ. ~ 

The Hamming weight w(c) of a code vector c is defined as the number of nonzero 
elements in the code vector. Equivalently, we may state that the Hamming weight of a 
code vector is the distance between the code vector and the all-zero code vector. 

The minimum distance di, of a linear block code is defined as the smallest Hamming 
distance between any pair of code vectors in the code. That is, the minimum distance is 
the same as the smallest Hamming weight of the difference between any pair of code 
vectors. From the closure property of linear block codes, the sum (or difference) of two 
code vectors is another code vector. Accordingly, we may state that the minimum distance 
of a linear block code is the smallest Hamming weight of the nonzero code vectors in the 
code. 

The minimum distance dpi, is related to the structure of the parity-check matrix H 
of the code in a fundamental way. From Equation (10.16) we know that a linear block 
code is defined by the set of all code vectors for which cH” = 0, where HT is the transpose 
of the parity-check matrix H. Let the matrix H be expressed in terms of its columns as 
follows: 


H = [h,, hp... hy] (10.24) 


Then, for a code vector c to satisfy the condition cH? = 0, the vector c must have 1s in 
such positions that the corresponding rows of H” sum to the zero vector 0. However, by 
definition, the number of 1s in a code vector is the Hamming weight of the code vector. 
Moreover, the smallest Hamming weight of the nonzero code vectors in a linear block 
code equals the minimum distance of the code. Hence, the minimum distance of a linear 
block code is defined by the minimum number of rows of the matrix HT whose sum is 
equal to the zero vector. 

The minimum distance of a linear block code, dmin is an important parameter of the 
code, Specifically, it determines the error-correcting capability of the code. Suppose an 
(n, k) linear block code is required to detect and correct all error patterns (over a binary 
symmetric channel), and whose Hamming weight is less than or equal to t. That is, if a 
code vector c; in the code is transmitted and the received vector is r = c; + e, we require 
that the decoder output ĉ = c;, whenever the error pattern e has a Hamming weight 
w(e) = t. We assume that the 2* code vectors in the code are transmitted with equal 
probability. The best strategy for the decoder then is to pick the code vector closest to the 
received vector r, that is, the one for which the Hamming distance d(c,, r) is the smallest. 
With such a strategy, the decoder will be able to detect and correct all error patterns of 
Hamming weight w(e) = t, provided that the minimum distance of the code is equal to or 
greater than 2t + 1. We may demonstrate the validity of this requirement by adopting a 
geometric interpretation of the problem. In particular, the 1-by-n code vectors and the 
1-by-n received vector are represented as points in an #-dimensional space. Suppose that 
we construct two spheres, each of radius t, around the points that represent code vectors 
c; and c;. Let these two spheres be disjoint, as depicted in Figure 10.62. For this condition 
to be satisfied, we require that d(c,, c;) = 2t + 1. If then the code vector c; is transmitted 
and the Hamming distance d(c;, r) = #, it is clear that the decoder will pick c; as it is the 


638 CHAPTER 10 & ERROR-CONTROL CODING 


&) Z Clg 
y 
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Ficure 10.6 (a) Hamming distance d(c;, ¢;) = 2t + 1. (b) Hamming distance d(c;, c;) < 2t, 
The received vector is denoted by r. 





code vector closest to the received vector r. If, on the other hand, the Hamming distance 
d(c;, c;) = 2t, the two spheres around c; and c;intersect, as depicted in Figure 10.6b. Here 
we see that if c; is transmitted, there exists a received vector r such that the Hamming 
distance d(c; r) = t, and yet r is as close to c; as it is to c;. Clearly, there is now the 
possibility of the decoder picking the vector c;, which is wrong. We thus conclude that an 
(n, k) linear block code has the power to correct all error patterns of weight t or less if, 
and only if, 


d(c c) = 2t+1 for all c; and c; 


By definition, however, the smallest distance between any pair of code vectors in a code 
is the minimum distance of the code, dmin We may therefore state that an (n, k) linear 
block code of minimum distance'dmin can correct up to t errors if, and only if, 


t =|4(dmin - 1) (10.25) 


where | | denotes the largest integer less than or equal to the enclosed quantity. Equation 
(10.25) gives the error-correcting capability of a linear block code a quantitative meaning. 


SYNDROME DECODING 


We are now ready to describe a syndrome-based decoding scheme for linear block codes. 
Let c1, €2,... 5 Cx denote the 2* code vectors of an (n, k) linear block code. Let r denote 
the received vector, which may have one of 2” possible values. The receiver has the task 
of partitioning the 2” possible received vectors into 2* disjoint subsets D1, Do, +. . , Da in 
such a way that the ith subset ®; corresponds to code vector c; for 1 = i = 24. The received 
vector r is decoded into c; if it is in the ¿th subset. For the decoding to be correct, r must 
be in the subset that belongs to the code vector c; that was actually sent. 

The 2* subsets described herein constitute a standard array of the linear block code. 
To construct it, we may exploit the linear structure of the code by proceeding as follows: 


1. The 2* code vectors are placed in a row with the all-zero code vector c, as the left- 
most element. 

2. An error pattern e is picked and placed under c,, and a second row is formed by 
adding e; to each of the remaining code vectors in the first row; it is important that 
the error pattern chosen as the first element in a row not have previously appeared 
in the standard array. 

3. Step 2 is repeated until all the possible error patterns have been accounted for. 


Figure 10.7 illustrates the structure of the standard array so constructed. The 2* columns 
of this array represent the disjoint subsets D1, D2, ..., Dx. The 2°-* rows of the array 
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FIGURE 10.7 Standard array for an (n, k) block code. 


represent the cosets of the code, and their first elements e2,..., eyr-# are called coset 
leaders. 

For a given channel, the probability of decoding error is minimized when the most 
likely error patterns (i.e., those with the largest probability of occurrence) are chosen as 
the coset leaders. In the case of a binary symmetric channel, the smaller the Hamming 
weight of an error pattern the more likely it is to occur. Accordingly, the standard array 
should be constructed with each coset leader having the minimum Hamming weight in its 
coset. 

We may now describe a decoding procedure for a linear block code: 


1. For the received vector r, compute the syndrome s = rH’, 

2. Within the coset characterized by the syndrome s, identify the coset leader (i.e., the 
error pattern with the largest probability of occurrence); call it eo. 

3. Compute the code vector 


c=r +e (10.26) 
as the decoded version of the received vector r. 


This procedure is called syndrome decoding. 


& EXAMPLE 10.2 Hamming Codes‘ 

Consider a family of (n, k) linear block codes that have the following parameters: 
Block length: n=2"-1 
Number of message bits: k = 2” — m- 1 
Number of parity bits: n-~k=m 

where m = 3. These are the so-called Hamming codes. 

Consider, for example, the (7, 4) Hamming code with n = 7 and k = 4, corresponding 
to m = 3. The generator matrix of the code must have a structure that conforms to Equation 


(10.12). The following matrix represents an appropriate generator matrix for the (7, 4) Ham- 
ming code: 


oF m m 
eee oo 
= ay = oe 
oo e. o 
or 0o o 
=. ooo 


640 CHAPTER 10 2 ERROR-CONTROL CODING 


TABLE 10.1 Code words of a (7, 4) Hamming code 



















Message Weight of Message Weight of 
Word Code Word Code Word Word Code Word Code Word 
0000000 0 1000 1101000 3 
0001 1010001 3 1001 0111001 4 
0010 1110010 4 1010 0011010 3 
0011 0100011 3 1011 1001011 4 
0100 0110100 3 1100 1011100 4 
0101 1100101 4 1101 0001101 3 
0110 1000110 3 1110 0101110 4 
0111 0010111 4 1111 1111111 7 





The corresponding parity-check matrix is given by 


1 00:1 011 
H=/0 1 0:1 110 

001:01141 

— 

L-z pr 


With k = 4, there are 2* = 16 distinct message words, which are listed in Table 
10.1. For a given message word, the corresponding code word is obtained by using Equa- 
tion (10.13). Thus, the application of this equation results in the 16 code words listed in 
Table 10.1. 

In Table 10.1, we have also listed the Hamming weights of the individual code words 
in the (7, 4) Hamming code. Since the smallest of the Hamming weights for the nonzero code 
words is 3, it follows that the minimum distance of the code is 3. Indeed, Hamming codes 
have the property that the minimum distance dmin = 3, independent of the value assigned to 
the number of parity bits m. 

To illustrate the relation between the minimum distance d,,;, and the structure of the 
parity-check matrix H, consider the code word 0110100. In the matrix multiplication defined 
by Equation (10.16), the nonzero elements of this code word “sift” out the second, third, and 
fifth columns of the matrix H yielding 


0 0 0 0 
1}+]O0/+ ]/1)/=]0 
0 1 1 0 


We may perform similar calculations for the remaining 14 nonzero code words. We thus find 
that the smallest number of columns in H that sums to zero is 3, confirming the earlier state- 
ment that dmin = 3. 

An important property of Hamming codes is that they satisfy the condition of Equation 
(10.25) with the equality sign, assuming that ¢ = 1. This means that Hamming codes are 
single-error correcting binary perfect codes. 

Assuming single-error patterns, we may formulate the seven coset leaders listed in the 
right-hand column of Table 10.2: The corresponding 2° syndromes, listed in the left-hand 
column, are calculated in accordance with Equation (10.20). The zero syndrome signifies no 
transmission errors. 

Suppose, for example, the code vector [1110010] is sent, and the received vector is 
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= TABLE 10.2 Decoding 
| table for the (7, 4) 
Hamming code defined 
© in Table 10.1 





Syndrome Error Pattern 





000 0000000 
100 1000000 
010 0100000 
001 0010000 
110 0001000 
011 0000100 
111 0000010 
101 0000001 





[1100010] with an error in the third bit. Using Equation (10.19), the syndrome is calculated 


to be 
100 
010 
00 1 
s = [1100010]} 1 1 0 
0 1 1 
111 
1041 


=[0 0 1) 


From Table 10.2 the corresponding coset leader (i.e., error pattern with the highest probability 
of occurrence) is found to be [0010000], indicating correctly that the third bit of the received 
vector is erroneous. Thus, adding this error pattern to the received vector, in accordance with 
Equation (10.26), yields the correct code vector actually sent. d 


@ DUAL CODE 


Given a linear block code, we may define its dual as follows. Taking the transpose of both 
sides of Equation (10.15), we have 


GH” = 0 
where H” is the transpose of the parity-check matrix of the code, and 0 is a new zero 
matrix. This equation suggests that every (n, k) linear block code with generator matrix 


G and parity-check matrix H has a dual code with parameters (n, n — k), generator matrix 
H and parity-check matrix G. 


110.4 Cyclic Codes 


Cyclic codes form a subclass of linear block codes. Indeed, many of the important linear 
block codes discovered to date are either cyclic codes or closely related to cyclic codes. An 
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advantage of cyclic codes over most other types of codes is that they are easy to encode. 
Furthermore, cyclic codes possess a well-defined mathematical structure, which has led to 
the development of very efficient decoding schemes for them. 
A binary code is said to be a cyclic code if it exhibits two fundamental Properties: 

1. Linearity property: The sum of any two code words in the code is also a code word, 

2. Cyclic property: Any cyclic shift of a code word in the code is also a code word, 
Property 1 restates the fact that a cyclic code is a linear block code (i.e., it can be described 
as a parity-check code): To restate Property 2 in mathematical terms, let the n-tuple 
(Co, C15 -- + » Cy-1) denote a code word of an (n, k) linear block code. The code is a cyclic 
code if the #-tuples 


(Cats Cop seas Cn—2)s 


(Cn-2s Cn—1s e.. Ca-3), 


(c1; C2926 +5 Cody Co) 


are all code words in the code. 
To develop the algebraic properties of cyclic codes, we use the elements Cos Ciye es, 
Cn—1 Of a code word to define the code polynomial 


cX) = co + GX + oX? +--+ + cp, aX (10.27) 


where X is an indeterminate. Naturally, for binary codes, the coefficients are 1s and 0s. 
Each power of X in the polynomial c(X) represents a one-bit shift in time. Hence, multi- 
plication of the polynomial c(X) by X may be viewed as a shift to the right. The key 
question is: How do we make such a shift cyclic? The answer to this question is addressed 
next. 
Let the code polynomial c(X) be multiplied by X’, yielding 
X'c(X) = Xoo + aX +-+ Cn-i-aX + 6g X 

tee + Gy X!) 
COX’ + aK + oe H Gy aX") + 6X" 
Fenit (ep, Cog a 
Cy iX” bee H Cu XE + OX! + aX 

+ SETE + AEO 


(10.28) 


where, in the last line, we have merely rearranged terms. Recognizing, for example, that 
Cn—i F C,-; = 0 in modulo-2 addition, we may manipulate the first i terms of Equation 
(10.28) as follows: 


XX) = cpi tee + py XP1 + Xi + XP 4.0.4 Cop g Xt 


£ (10.29) 
+ Cn; X” + 1) +... + tX" 1X" 4 1) 
Next, we introduce the following definitions: 
<i) = P aoe gasi i i+1 
OX) = Cai Hii H Cpa X! + coX! + X (10.30) 


Hoet ee X! 
QX) = cni + CpnaX +e + epa Xit (10.31) 
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Accordingly, Equation (10.29) is reformulated in the compact form 
Xic(X) = q XX” + 1) + c(X) (10.32) 


The polynomial c(X) is recognized as the code polynomial of the code word (ca-»..., 
Ca—15 Cos C1y + + +» Cx—-j-1) Obtained by applying ż cyclic shifts to the code word (co, G4, «+, 
Cn—i-ts Cniy + + + » Gy 1) Moreover, from Equation (10.32) we readily see that c(X) is the 
remainder that results from dividing X‘c(X) by (X7 + 1). We may thus formally state the 
cyclic property in polynomial notation as follows: If c(X) is a code polynomial, then the 
polynomial 


cX) = K'c(X) mod(X” + 1) (10.33) 


is also a code polynomial for any cyclic shift i; the term mod is the abbreviation for modulo. 
The special form of polynomial multiplication described in Equation (10.33) is referred to 
as multiplication modulo X” + 1. In effect, the multiplication is subject to the constraint 
X” = 1, the application of which restores the polynomial X‘c(X) to order n — 1 for all 
i < n. (Note that in modulo-2 arithmetic, X” + 1 has the same value as X” — 1.) 


GENERATOR POLYNOMIAL 


The polynomial X* + 1 and its factors play a major role in the generation of cyclic codes. 
Let g(X) be a polynomial of degree n — k that is a factor of X” + 1; as such, g(X) is the 
polynomial of least degree in the code. In general, g(X) may be expanded as follows: 
n—k-1 
g(X)=1+ È gXi+ xr (10.34) 
i=1 

where the coefficient g; is equal to 0 or 1. According to this expansion, the polynomial 
g(X) has two terms with coefficient 1 separated by n — k — 1 terms. The polynomial g(X) 
is called the generator polynomial of a cyclic code. A cyclic code is uniquely determined 
by the generator polynomial g(X) in that each code polynomial in the code can be ex- 
pressed in the form of a polynomial product as follows: 


c(X) = a(X)g(X) (10.35) 


where a(X) is a polynomial in X with degree k — 1. The c(X) so formed satisfies the 
condition of Equation (10.33) since g(X) is a factor of X” + 1. 

Suppose we are given the generator polynomial g(X) and the requirement is to encode 
the message sequence (mo, Ma, - - - , Mg—1) into an (n, k) systematic cyclic code. That is, 
the message bits are transmitted in unaltered form, as shown by the following structure 
for a code word (see Figure 10.4): 


(bo, bis.. -p bDu-k-1> Mo, Mis... Mp1) 
CARE 
n — k parity bits k message bits 


Let the message polynomial be defined by 

m(X) = mo + mX +-+- + m, Xt! (10.36) 
and let 

b(X) = bo + bX +--+ + by XA (10.37) 
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According to Equation (10.1), we want the code polynomial to be in the form 
(X) = b(X) + X”*n(X) (10.38) 
Hence, the use of Equations (10.35) and (10.38) yields 
a(X)g(X) = b(X) + X*-*m(X) 
Equivalently, in light of modulo-2 addition, we may write 
f X” *m(X) b(X) 
g(x) g(x) 


Equation (10.39) states that the polynomial b(X) is the remainder left over after dividing 
X”-tm(X) by g(X). 

We may now summarize the steps involved in the encoding procedure for an (x, k) 
cyclic code assured of a systematic structure. Specifically, we proceed as follows: 


= aX) + (10.39) 


1. Multiply the message polynomial m(X) by X*™*. 
2. Divide X"~*m(X) by the generator polynomial g(X), obtaining the remainder b(X), 
3. Add b(X) to X*-*m(X), obtaining the code polynomial c(X). 


m Parity-CHECK POLYNOMIAL 


An (n, k) cyclic code is uniquely specified by its generator polynomial g(X) of order 
(n — k). Such a code is also uniquely specified by another polynomial of degree k, which 
is called the parity-check polynomial, defined by 

k-1 


A(X) = 1+ > hX’ + X* (10.40) 
i=1 


where the coefficients h; are 0 or 1. The parity-check polynomial 4(X) has a form similar 
to the generator polynomial in that there are two terms with coefficient 1, but separated 
by & — 1 terms. 

The generator polynomial g(X) is equivalent to the generator matrix G as a descrip- 
tion of the code. Correspondingly, the parity-check polynomial, denoted by 4(X), is an 
equivalent representation of the parity-check matrix H. We thus find that the matrix re- 
lation HGT = 0 presented in Equation (10.15) for linear block codes corresponds to the 
relationship 


g(X)h(X) mod(X” + 1) = 0 (10.41) 


Accordingly, we may state that the generator polynomial g(X) and the parity-check poly- 
nomial h(X) are factors of the polynomial X” + 1, as shown by 


g(X)b(X) =X" +1 (10.42) 


This property provides the basis for selecting the generator or parity-check polynomial of 
a cyclic code. In particular, we may state that if g(X) is a polynomial of degree (n— k) 
and it is also a factor of X”+ 1, then g(X) is the generator polynomial of an (x, k) cyclic 
code. Equivalently, we may state that if A(X) is a polynomial of degree k and it is also a 
factor of X” + 1, then h(X) is the parity-check polynomial of an (1, k) cyclic code. 

A final comment is in order. Any factor of X” + 1 with degree (1 — k), the number 
of parity bits, can be used as a generator polynomial. For large values of n, the polynomial 
X” + 1 may have many factors of degree n — k. Some of these polynomial factors generate 
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good cyclic codes, whereas some of them generate bad cyclic codes. The issue of how to 
select generator polynomials that produce good cyclic codes is very difficult to resolve. 
Indeed, coding theorists have expended much effort in the search for good cyclic codes. 


GENERATOR AND PARITY-CHECK MATRICES 


Given the generator polynomial g(X) of an (n, k) cyclic code, we may construct the gen- 
erator matrix G of the code by noting that the k polynomials g(X), Xg(X), . . . , X*~'g(X) 
span the code. Hence, the n-tuples corresponding to these polynomials may be used as 
rows of the k-by-n generator matrix G. 

However, the construction of the parity-check matrix H of the cyclic code from the 
parity-check polynomial 4(X) requires special attention, as described here. Multiplying 
Equation (10.42) by a(x) and then using Equation (10.35), we obtain 


c(X)b(X) = a(X) + X%a(X) (10.43) 


The polynomials c(X) and 4(X) are themselves defined by Equations (10.27) and (10.40), 
respectively, which means that their product on the left-hand side of Equation (10.43) 
contains terms with powers extending up to n + k — 1. On the other hand, the polynomial 
a(X) has degree k — 1 or less, the implication of which is that the powers of X*, X**1,..., 
X”! do not appear in the polynomial on the right-hand side of Equation (10.43). Thus, 
setting the coefficients of X*, X*~1,..., X"~1 in the expansion of the product polynomial 
c(X)h(X) equal to zero, we obtain the following set of mn’ — k equations: 

j+k 

È chesj-i= 0 forOsjsn-k-1 (10.44) 

=] 
Comparing Equation (10.44) with the corresponding relation of Equation (10.16), we may 
make the following important observation: The coefficients of the parity-check polynomial 
h(X) involved in the polynomial multiplication described in Equation (10.44) are arranged 
in reversed order with respect to the coefficients of the parity-check matrix H involved in 
forming the inner product of vectors described in Equation (10.16). This observation sug- 
gests that we define the reciprocal of the parity-check polynomial as follows: 


k-1 
X*h(X~1) = x(a +5 bX” + x=) 
Ai (10.45) 


k-1 


=1+ X hX + X* 
i=1 


which is also a factor of X” + 1. The n-tuples pertaining to the (n — k) polynomials 
X*h(X~*), X**1h(X—4), ..., X"-14(X—1) may now be used in rows of the ( — k)-by-n 
parity-check matrix H. 

In general, the generator matrix G and the parity-check matrix H constructed in the 
manner described here are not in their systematic forms. They can be put into their sys- 
tematic forms by performing simple operations on their respective rows, as illustrated in 
Example 10.3. 


ENCODER FOR CYCLIC CODES 


Earlier we showed that the encoding procedure for an (n, k) cyclic code in systematic form 
involves three steps: (1) multiplication of the message polynomial 7(X) by X”~, (2) di- 
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FIGURE 10.8 Encoder for an (n, k) cyclic code. 


vision of X”~*m(X) by the generator polynomial g(X) to obtain the remainder b(X), and 
(3) addition of b(X) to X”-*m(X) to form the desired code polynomial. These three steps 
can be implemented by means of the encoder shown in Figure 10.8, consisting of a linear 
feedback shift register with (n — k) stages. 

The boxes in Figure 10.8 represent flip-flops, or unit-delay elements. The flip-flop is 
a device that resides in one of two possible states denoted by 0 and 1. An external clock 
(not shown in Figure 10.8) controls the operation of all the flip-flops. Every time the clock 
ticks, the contents of the flip-flops (initially set to the state 0) are shifted out in the direction 
of the arrows. In addition to the flip-flops, the encoder of Figure 10.8 includes a second 
set of logic elements, namely, adders, which compute the modulo-2 sums of their respective 
inputs. Finally, the multipliers multiply their respective inputs by the associated coeffi- 
cients. In particular, if the coefficient g; = 1, the multiplier is just a direct “connection.” 
If, on the other hand, the coefficient g; = 0, the multiplier is “no connection.” 

The operation of the encoder shown in Figure 10.8 proceeds as follows: 


1. The gate is switched on. Hence, the k message bits are shifted into the channel, As 
soon as the k message bits have entered the shift register, the resulting (n — k) bits 
in the register form the parity bits [recall that the parity bits are the same as the 
coefficients of the remainder b(X)]. 

2. The gate is switched off, thereby breaking the feedback connections. 

3. The contents of the shift register are read out into the channel. 


CALCULATION OF THE SYNDROME 


Suppose the code word (co, ¢1, .. - 5 Cn—1) is transmitted over a noisy channel, resulting in 
the received word (19, 71, . . 5 74-1). From Section 10.3, we recall that the first step in the 
decoding of a linear block code is to calculate the syndrome for the received word. If the 
syndrome is zero, there are no transmission errors in the received word. If, on the other 
hand, the syndrome is nonzero, the received word contains transmission errors that require 
correction. 

In the case of a cyclic code in systematic form, the syndrome can be calculated easily. 
Let the received word be represented by a polynomial of degree n — 1 or less, as shown 


by 
HX) = ro +X tees t+ r, aX! (10.46) 
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Let q(X) denote the quotient and s(X) denote the remainder, which are the results of 
dividing r(X) by the generator polynomial g(X). We may therefore express r(X) as follows: 


r(X) = q(X)g(X) + s(X) (10.47) 


The remainder s(X) is a polynomial of degree n — k — 1 or less, which is the result of 
interest, It is called the syndrome polynomial because its coefficients make up the (x — k)- 
by-1 syndrome s. . 

Figure 10.9 shows a syndrome calculator that is identical to the encoder of Figure 
10.8 except for the fact that the received bits are fed into the (# — k) stages of the feedback 
shift register from the left. As soon as all the received bits have been shifted into the shift 
register, its contents define the syndrome s. 

The syndrome polynomial s(X) has the following useful properties that follow from 
the definition given in Equation (10.47). 


1. The syndrome of a received word polynomial is also the syndrome of the corresponding 
error polynomial. 


Given that a cyclic code with polynomial c(X) is sent over a noisy channel, the received 
word polynomial is defined by 


7(X) = (X) + e(X) (10.48) 
where e(X) is the error polynomial. Equivalently, we may write 
e(X) = (X) + c(X) (10.49) 
Hence, substituting Equations (10.35) and (10.47) into (10.49), we get 
e(X) = u(X)g(X) + s(X) (10.50) 


where the quotient is u(X) = a(X) + q(X). Equation (10.50) shows that s(X) is also the 
syndrome of the error polynomial e(X). The implication of this property is that when the 
syndrome polynomial s(X) is nonzero, the presence of transmission errors in the received 
word is detected. 


2. Let s(X) be the syndrome of a received word polynomial r(X). Then, the syndrome of 
Xr(X), a cyclic shift of r(X), is Xs(X). 
Applying a cyclic shift to both sides of Equation (10.47), we get 

Xr(X) = Xq(X)g(X) + Xs(X) (10.51) 








Flip-flop Modulo-2 
adder 


FIGURE 10.9 Syndrome calculator for (n, k) cyclic code. 
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from which we readily see that Xs(X) is the remainder of the division of Xr(X) by (X). 
Hence, the syndrome of Xr(X) is Xs(X) as stated. We may generalize this result by stating 
that if s(X) is the syndrome of 7(X), then X’s(X) is the syndrome of X’r(X). 


3. The syndrome polynomial s(X) is identical to the error polynomial e(X), assuming that 
the errors are confined to the (n — k) parity-check bits of the received word polynomial 
r(X). 

The assumption made here is another way of saying that the degree of the error polynomial 
e(X) is less than or equal to (n — k — 1). Since the generator polynomial g(X) is of degree 
(n — k), by definition, it follows that Equation (10.50) can only be satisfied if the quotient 
u(X) is zero. In other words, the error polynomial e(X) and the syndrome polynomial s(x) 
are one and the same. The implication of Property 3 is that, under the aforementioned 
conditions, error correction can be accomplished simply by adding the syndrome poly- 
nomial s(X) to the received word polynomial r(X). 


» EXAMPLE 10.3 Hamming Codes Revisited 


To illustrate the issues relating to the polynomial representation of cyclic codes, we consider 
the generation of a (7, 4) cyclic code. With the block length n = 7, we start by factorizing 
X’ + 1 into three irreducible polynomials: 
X74+1= (1+ X)1 + X? + X91 + X + X3) 

By an “irreducible polynomial” we mean a polynomial that cannot be factored using only 
polynomials with coefficients from the binary field. An irreducible polynomial of degree 
m is said to be primitive if the smallest positive integer n for which the polynomial divides 
X” + 1 isn = 2” — 1. For the example at hand, the two polynomials (1 + X? + X°) and 
(1 + X + X?) are primitive. Let us take 

g(X)=1+X+ xX? 
as the generator polynomial, whose degree equals the number of parity bits. This means that 
the parity-check polynomial is given by 

h(X) = (1 + X)(1 +X? + X’) 

=14+X4+X24 x! 
whose degree equals the number of message bits k = 4. 

Next, we illustrate the procedure for the construction of a code word by using this 
generator polynomial to encode the message sequence 1001. The corresponding message poly- 
nomial is given by 

mX) =1+X? 
Hence, multiplying m(X) by X4 = X’, we get 
X"#en(X) = X? + X6 
The second step is to divide X”~*m(X) by g(X), the details of which (for the example at hand) 
are given below: 
X? +X 
X3 +X + 1)x® +X’ 
xé +X + X? 
xí 
x? +X? +X 
X’ +X 
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Note that in this long division we have treated subtraction the same as addition, since we are 
operating in modulo-2 arithmetic. We may thus write 
X? + X$ X + xX? 
ose EX 4K 4+ ]7 
1+xX+x 1+xX+x 


That is, the quotient a(X) and remainder b(X) are as follows, respectively: 
aX) =X + X 
b(X) = X + X? 
Hence, from Equation (10.38) we find that the desired code polynomial is 
X) = b(X) + X*-*m(X) 
=X + X* +X? +X" 
The code word is therefore 0111001. The four right-most bits, 1001, are the specified message 
bits. The three left-most bits, 011, are the parity-check bits. The code word thus generated is 
exactly the same as the corresponding one shown in Table 10.1 for a (7, 4) Hamming code. 
We may generalize this result by stating that any cyclic code generated by a primitive 
polynomial is a Hamming code of minimum distance 3. 
i We next show that the generator polynomial g(X) and the parity-check polynomial 
h(X) uniquely specify the generator matrix G and the parity-check matrix H, respectively. 
To construct the 4-by-7 generator matrix G, we start with four polynomials represented 
by g(X) and three cyclic-shifted versions of it, as shown by 
g(X)=14+X+X° 
Xg(X) = X + X + X* 
X?g(X) = X* + X? + X5 
X3g(X) = X? + X4 + X5 
The polynomials g(X), Xg(X), X?g(X), and X°g(X) represent code polynomials in the (7, 4) 
Hamming code. If the coefficients of these polynomials are used as the elements of the rows 
of a 4-by-7 matrix, we get the following generator matrix: 


1101000 
0110100 
G = 
0011010 
0001101 


Clearly, the generator matrix G’ so constructed is not in systematic form. We can put it into 
a systematic form by adding the first row to the third row, and adding the sum of the first 
two rows to the fourth row. These manipulations result in the desired generator matrix: 


1101000 

0110100 
G= 

1 110010 

1010001 


which is exactly the same as that in Example 10.2. 

We next show how to construct the 3-by-7 parity-check matrix H from the parity-check 
polynomial /(X). To do this, we first take the reciprocal of b(X), namely, X*h(X—"). For the 
problem at hand, we form three polynomials represented by X*h(X~') and two shifted ver- 
sions of it, as shown by 

X'h X’) = 1+ X? +X? + Xt 
Xhi X) = X + X + X*4+ X 
XŚÉh X!) = X + X+ X + X® 
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Figure 10.10 Encoder for the (7, 4) cyclic code generated by g(X) = 1 + X + X’. 


Using the coefficients of these three polynomials as the elements of the rows of the 3-by-7 
parity-check matrix, we get 


H= 


=e o 


0 
0 
1 


oo m 
oro 
=. OF 
me e e 


1 
1 
0 


Here again we see that the matrix H’ is not in systematic form. To put it into a systematic 
form, we add the third row to the first row to obtain 


10031011 
H=/0 10111 0 
0010111 


which is exactly the same as that of Example 10.2. 

Figure 10.10 shows the encoder for the (7, 4) cyclic Hamming code generated by the 
polynomial g(X) = 1 + X + X’. To illustrate the operation of this encoder, consider the 
message sequence (1001). The contents of the shift register are modified by the incoming 
message bits as in Table 10.3. After four shifts, the contents of the shift register, and therefore 
the parity bits, are (011). Accordingly, appending these parity bits to the message bits (1001), 
we get the code word (0111001); this result is exactly the same as that determined earlier in 
the example. 

Figure 10.11 shows the corresponding syndrome calculator for the (7, 4) Hamming 
code. Let the transmitted code word be (0111001) and the received word be (0110001); that 
is, the middle bit is in error. As the received bits are fed into the shift register, initially set to 
zero, its contents are modified as in Table 10.4. At the end of the seventh shift, the syndrome 
is identified from the contents of the shift register as 110. Since the syndrome is nonzero, the 
received word is in error. Moreover, from Table 10.2, we see that the error pattern corre- 
sponding to this syndrome is 0001000. This indicates that the error is in the middle bit of the 
received word, which is indeed the case. 4 


TABLE 10.3 Contents of the 
shift register in the encoder 
of Figure 10.10 for message 
sequence (1001) 





Input Register Contents 





PWN 


0 0 0 (initial state) 
110 
011 
111 
011 


eoor 
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Figure 10,11 Syndrome calculator for the (7, 4) cyclic code generated by the polynomial 
g(X)=14+X4X3. 


œ EXAMPLE 10.4 Maximal-Length Codes 


For any positive integer m = 3, there exists a maximal-length code with the following 


parameters: 
Block length: n=2"—1 
Number of message bits: kR=m 
Minimum distance: doin = 274 


Maximal-length codes are generated by polynomials of the form 


1+ X” 


g(X) = AX) 





(10.52) 


where b(X) is any primitive polynomial of degree m. Earlier we stated that any cyclic code 
generated by a primitive polynomial is a Hamming code of minimum distance 3 (see Example 
10.3). It follows therefore that maximal-length codes are the dual of Hamming codes. 

The polynomial h(X) defines the feedback connections of the encoder. The generator 
polynomial g(X) defines one period of the maximal-length code, assuming that the encoder is 
in the initial state 00 ... 01. To illustrate these points, consider the example of a (7, 3) 
maximal-length code, which is the dual of the (7, 4) Hamming code described in Example 
10.3. Thus, choosing 


(X)=14+X%+ X’ 
we find that the generator polynomial of the (7, 3) maximal-length code is 


g(X)=1+X +X? +X’ 


2 TABLE 10.4 Contents of the syndrome 
calculator in Figure 10.11 for the 
received word 0110001 





Shift Input Bit Contents of Shift Register 





0 0 0 (initial state) 


1 1 100 
2 0 010 
3 0 001 
4 0 110 
5 1 111 
6 1 001 
7 0 110 
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FIGURE 10.12 Encoder for the (7, 3) maximal-length code; the initial state of the encoder is 
shown in the figure. 


Figure 10.12 shows the encoder for the (7, 3) maximal-length code, the feedback connections 
of which are exactly the same as those shown in Figure 8.2 in Chapter 8. The period of the 
code is n = 7. Thus, assuming that the encoder is in the initial state 001, as indicated in Figure 
10.12, we find the output sequence is described by 





100 14110100 
initial «© g(X) | 1 + X + X2 + X* 
state 


This result may be readily validated by cycling through the encoder of Figure 10.12. 
Note that if we were to choose the other primitive polynomial 


h(X) = 1+ X +X’ 


for the (7, 3) maximal-length code, we would simply get the “image” of the code described 
above, and the output sequence would be “reversed” in time. 4 


a OTHER CYCLIC CODES 


We conclude the discussion of cyclic codes by presenting the characteristics of three other 
important classes of cyclic codes. 


Cyclic Redundancy Check Codes 


Cyclic codes are extremely well-suited for error detection, We make this statement 
for two reasons. First, they can be designed to detect many combinations of likely errors. 
Second, the implementation of both encoding and error-detecting circuits is practical. It is 
for these reasons that many of the error-detecting codes used in practice are of the cyclic- 
code type. A cyclic code used for error-detection is referred to as cyclic redundancy check 
(CRC) code. 

We define an error burst of length B in an n-bit received word as a contiguous 
sequence of B bits in which the first and last bits or any number of intermediate bits are 
received in error. Binary (7, k) CRC codes are capable of detecting the following error 
patterns: 


1. All error bursts of length 7# — k or less. 
2. A fraction of error bursts of length equal to n — k + 1; the fraction equals 


1-2-6), 
3. A fraction of error bursts of length greater than n — k + 1; the fraction equals 
jhe RER 


4, All combinations of d,i, — 1 (or fewer) errors. 
5. All error patterns with an odd number of errors if the generator polynomial g(X) 
for the code has an even number of nonzero coefficients. 
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| TABLE 10.5 CRC codes 





Code Generator Polynomial, 2(X) n-k 
CRC-12 code 1+X+X? +X? +X" +X’ 12 
CRC-16 code (USA) 1 +X? +X" +x" 16 
CRC-ITU code 1+ X5 + Xx + x" 16 





Table 10.5 presents the generator polynomials of three CRC codes that have become 
international standards. All three codes contain 1 + X as a prime factor. The CRÇ-12 
code is used for 6-bit characters, and the other two codes are used for 8-bit characters. 
CRC codes provide a powerful method of error detection for use in automatic-repeat 
request (ARQ) strategies discussed in Section 10.1, and digital subscriber lines discussed 
in Chapter 4. 


Bose—Chaudhuri—Hocquenghem (BCH) Codes? 


One of the most important and powerful classes of linear-block codes are BCH codes, 
which are cyclic codes with a wide variety of parameters. The most common binary BCH 
codes, known as primitive BCH codes, are characterized for any positive integers m (equal 
to or greater than 3) and ¢ [less than (2” — 1)/2] by the following parameters: 


Block length: n=2™—1 
Number of message bits: k = n — mt 
Minimum distance: Amin Z 2t +1 


Each BCH code is a t-error correcting code in that it can detect and correct up to t random 
errors per code word. The Hamming single-error correcting codes can be described as 
BCH codes. The BCH codes offer flexibility in the choice of code parameters, namely, 
block length and code rate. Furthermore, for block lengths of a few hundred bits or less, 
the BCH codes are among the best known codes of the same block length and code rate. 

A detailed treatment of the construction of BCH codes is beyond the scope of 
our present discussion. To provide a feel for their capability, we present in Table 10.6, the 
code parameters and generator polynomials for binary block BCH codes of length up to 
2° — 1, For example, suppose we wish to construct the generator polynomial for (15, 7) 


§ TABLE 10.6 Binary BCH codes of length up to 2° — 1 








n k t Generator Polynomial 

7 4 1 1 011 
15 11 1 10 011 
15 7 2 111 010 001 
15 5 3 10 100 110 111 
31 26 1 100 101 
31 21 2 11 101 101 001 
31 16 3 1 000 111 110 101 111 
31 11 5 101 100 010 011 011 010 101 
31 6 7 11 001 011 011 110 101 000 100 111 





Notation: n = block length 
k = number of message bits 
t = maximum number of detectable errors 


The high-order coefficients of the generator polynomial g(X) are at the left. 
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BCH code. From Table 10.6 we have (111 010 001) for the coefficients of the generator 
polynomial; hence, we write 


g(X) =X +X +X +X’+1 


Reed-Solomon Codes“ 


The Reed-Solomon codes are an important subclass of nonbinary BCH codes; they 
are often abbreviated as RS codes. The encoder for an RS code differs from a binary 
encoder in that it operates on multiple bits rather than individual bits. Specifically, an RS 
(n, k) code is used to encode m-bit symbols into blocks consisting ofn = 2” — 1 symbols, 
that is, m(2” — 1) bits, where m = 1. Thus, the encoding algorithm expands a block of k 
symbols to 7 symbols by adding n — k redundant symbols. When 7n is an integer power 
of two, the m-bit symbols are called bytes. A popular value of m is 8; indeed, 8-bit RS 
codes are extremely powerful. 

A f-error-correcting RS code has the following parameters: 


Block length: n = 2" — 1 symbols 
Message size: k symbols 
Parity-check size: n — k = 2t symbols 


Minimum distance: dain = 2t + 1 symbols 


The block length of the RS code is one less than the size of a code symbol, and the minimum 
distance is one greater than the number of parity-check symbols. The RS codes make highly 
efficient use of redundancy, and block lengths and symbol sizes can be adjusted readily to 
accommodate a wide range of message sizes. Moreover, the RS codes provide a wide range 
of code rates that can be chosen to optimize performance. Finally, efficient decoding tech- 
niques are available for use with RS codes, which is one more reason for their wide ap- 
plication (e.g., compact disc digital audio systems). 


i 10.5 Convolutional Codes’ 


In block coding, the encoder accepts a k-bit message block and generates an n-bit code 
word. Thus, code words are produced on a block-by-block basis. Clearly, provision must 
be made in the encoder to buffer an entire message block before generating the associated 
code word. There are applications, however, where the message bits come in serially rather 
than in large blocks, in which case the use of a buffer may be undesirable. In such situa- 
tions, the use of convolutional coding may be the preferred method. A convolutional coder 
generates redundant bits by using modulo-2 convolutions, hence the name. 

The encoder of a binary convolutional code with rate 1/7, measured in bits per 
symbol, may be viewed as a finite-state machine that consists of an M-stage shift register 
with prescribed connections to n modulo-2 adders, and a multiplexer that serializes the 
outputs of the adders. An L-bit message sequence produces a coded output sequence of 
length n(L + M) bits. The code rate is therefore given by 


L 


a i 53 
AL + M) bits/symbol (10.53) 


r 
Typically, we have L >> M. Hence, the code rate simplifies to 


bits/symbol (10.54) 


ERA 
n 
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The constraint length of a convolutional code, expressed in terms of message bits, is defined 
as the number of shifts over which a single message bit can influence the encoder output. 
In an encoder with an M-stage shift register, the memory of the encoder equals M message 
bits, and K = M + 1 shifts are required for a message bit to enter the shift register and 
finally come out. Hence, the constraint length of the encoder is K. 

Figure 10.134 shows a convolutional encoder with n = 2-and K = 3. Hence, the 
code rate of this encoder is 1/2. The encoder of Figure 10.134 operates on the incoming 
message sequence, one bit at a time. f 

We may generate a binary convolutional code with rate k/n by using k separate shift 
registers with prescribed connections to n modulo-2 adders, an input multiplexer and 
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FIGURE 10.13 (a) Constraint length-3, rate-} convolutional encoder. (b) Constraint length-2, 
rate- convolutional encoder. 
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an output multiplexer. An example of such an encoder is shown in Figure 10.13) 
where k = 2, n = 3, and the two shift registers have K = 2 each. The code rate is 2/3, In 
this second example, the encoder processes the incoming message sequence two bits at a 
time. 

The convolutional codes generated by the encoders of Figure 10.13 are nonsystematic 
codes. Unlike block coding, the use of nonsystematic codes is ordinarily preferred over 
systematic codes in convolutional coding. 

Each path connecting the output to the input of a convolutional encoder may be 
characterized in terms of its impulse response, defined as the response of that path to a 
symbol 1 applied to its input, with each flip-flop in the encoder set initially in the zero 
state, Equivalently, we may characterize each path in terms of a generator polynomial, 
defined as the unit-delay transform of the impulse response. To be specific, let the generator 
sequence (gË, gË, g%,..., ght) denote the impulse response of the ith path, where the 
coefficients gi, gË, g,..., gi} equal 0 or 1. Correspondingly, the generator polynomial 
of the ith path is defined by 


g(D) = gË + gD + gD? +--+ + IDM (10.55) 


where D denotes the unit-delay variable. The complete convolutional encoder is described 
by the set of generator polynomials {g(D), g2(D), ..., g°(D)]. Traditionally, different 
yariables are used for the description of convolutional and cyclic codes, with D being 
commonly used for convolutional codes and X for cyclic codes. 


i EXAMPLE 10.5 


Consider the convolutional encoder of Figure 10.134, which has two paths numbered 1 and 
2 for convenience of reference. The impulse response of path 1 (i.e., upper path) is (1, 1, 1). 
Hence, the cortesponding generator polynomial is given by 


gD) =1+ D + D? 


The impulse response of path 2 (i.e., lower path) is (1, 0, 1). Hence, the corresponding gen- 
erator polynomial is given by 


g?(D) = 1+ D? 
For the message sequence (10011), say, we have the polynomial representation 
m(D) = 1+ D? + D* 


As with Fourier transformation, convolution in the time domain is transformed into multi- 
plication in the D-domain. Hence, the output polynomial of path 1 is given by 


e(D) = g™(D)m(D) 
(1+ D + D*{1 + D? + D*) 
=1+ D+ D? +D’ + D* 


il 


From this we immediately deduce that the output sequence of path 1 is (1111001). Similarly, 
the output polynomial of path 2 is given by 


cD) = g(D)m{D) 
= (1+ D?\(4 + D? + D’) 
=1 + D? + D? +D’ + D +D 
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The output sequence of path 2 is therefore (1011111). Finally, multiplexing the two output 
sequences of paths 1 and 2, we get the encoded sequence 


c = (11, 10, 11, 11, 01, 01, 11) 


Note that the message sequence of length L = 5 bits produces an encoded sequence of length 
n(L + K — 1) = 14 bits. Note also that for the shift register to be restored to its zero initial 
` state, a terminating sequence of K — 1 = 2 zeros is appended to the last input bit of the 


message sequence. The terminating sequence of K — 1 zeros is called the tail of the message. 
< 


= CODE TREE, TRELLIS, AND STATE DIAGRAM 


Traditionally, the structural properties of a convolutional encoder are portrayed in graph- 
ical form by using any one of three equivalent diagrams: code tree, trellis, and state dia- 
gram. We will use the convolutional encoder of Figure 10.134 as a running example to 
illustrate the insights that each one of these three diagrams can provide. 

We begin the discussion with the code tree of Figure 10.14. Each branch of the tree 
represents an input symbol, with the corresponding pair of output binary symbols indi- 
cated on the branch. The convention used to distinguish the input binary symbols 0 and 
1 is as follows. An input 0 specifies the upper branch of a bifurcation, whereas input 1 
specifies the lower branch. A specific path in the tree is traced from left to right in accor- 
dance with the input (message) sequence. The corresponding coded symbols on the 
branches of that path constitute the input (message) sequence. Consider, for example, the 
message sequence (10011) applied to the input of the encoder of Figure 10.134. Following 
the procedure just described, we find that the corresponding encoded sequence is 
(11, 10, 11, 11, 01), which agrees with the first 5 pairs of bits in the encoded sequence {c;} 
derived in Example 10.5. 

From the diagram of Figure 10.14, we observe that the tree becomes repetitive after 
the first three branches. Indeed, beyond the third branch, the two nodes labeled a are 
identical, and so are all the other node pairs that are identically labeled. We may establish 
this repetitive property of the tree by examining the associated encoder of Figure 10.134. 
The encoder has memory M = K — 1 = 2 message bits. Hence, when the third message 
bit enters the encoder, the first message bit is shifted out of the register. Consequently, 
after the third branch, the message sequences (100 m3m,.. .) and (000 mm4 . . .) generate 
the same code symbols, and the pair of nodes labeled a may be joined together. The same 
reasoning applies to other nodes. Accordingly, we may collapse the code tree of Figure 
10.14 into the new form shown in Figure 10.15, which is called a #rellis.* It is so called 
since a trellis is a treelike structure with remerging branches. The convention used in Figure 
10.15 to distinguish between input symbols 0 and 1 is as follows. A code branch produced 
by an input 0 is drawn as a solid line, whereas a code branch produced by an input 1 is 
drawn as a dashed line. As before, each input (message) sequence corresponds to a specific 
path through the trellis. For example, we readily see from Figure 10.15 that the message 
sequence (10011) produces the encoded output sequence (11, 10, 11, 11, 01), which agrees 
with our previous result. A 

A trellis is more instructive than a tree in that it brings out explicitly the fact that 
the associated convolutional encoder is a finite-state machine. We define the state of a 
convolutional encoder of rate 1/7 as the (K — 1) message bits stored in the encoder’s shift 
register. At time j, the portion of the message sequence containing the most recent K bits 
is written as (M;-K+1s +--+ %;-1, m;), where m; is the current bit. The (K — 1)-bit state of 
the encoder at time j is therefore written simply as (m;—1, . - . , %j-K+2» Mj-xK+1). In the 
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FIGURE 10.14 Code tree for the convolutional encoder of Figure 10.132. 


case of the simple convolutional encoder of Figure 10.134 we have (K ~ 1) = 2. Hence, 
the state of this encoder can assume any one of four possible values, as described in Table 
10.7. The trellis contains (L + K) levels, where L is the length of the incoming message 
sequence, and K is the constraint length of the code. The levels of the trellis are la beled as 
j=0,1,...,L +K —1 in Figure 10.15 for K = 3. Level j is also referred to as depth j; 
both terms are used interchangeably. The first (K— 1) levels correspond to the encoder’s 
departure from the initial state a, and the last (K — 1) levels correspond to the encoder’s 
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Figure 10.15 Trellis for the convolutional encoder of Figure 10.13a. 


return to the state a. Clearly, not all the states can be reached in these two portions of the 
trellis. However, in the central portion of the trellis, for which the level 7 lies in the range 
K — 1 = j = L, all the states of the encoder are reachable. Note also that the central 
portion of the trellis exhibits a fixed periodic structure. 

Consider next a portion of the trellis corresponding to times j andj + 1. We assume 
that į = 2 for the example at hand, so that it is possible for the current state of the encoder 
to be a, b, c, or d. For convenience of presentation, we have reproduced this portion of 
the trellis in Figure 10.16a. The left nodes represent the four possible current states of the 
encoder, whereas the right nodes represent the next states. Clearly, we may coalesce the 
left and right nodes. By so doing, we obtain the state diagram of the encoder, shown in 
Figure 10,16. The nodes of the figure represent the four possible states of the encoder, 
with each node having two incoming branches and two outgoing branches. A transition 
from one state to another in response to input 0 is represented by a solid branch, whereas 
a transition in response to input 1 is represented by a dashed branch. The binary label on 
each branch represents the encoder’s output as it moves from one state to another. Suppose, 
for example, the current state of the encoder is (01), which is represented by node c. The 
application of input 1 to the encoder of Figure 10.13a results in the state (10) and the 
encoded output (00). Accordingly, with the help of this state diagram, we may readily 
determine the output of the encoder of Figure 10.13¢ for any incoming message sequence. 
We simply start at state a, the all-zero initial state, and walk through the state diagram in 
accordance with the message sequence. We follow a solid branch if the input is a 0 anda 
dashed branch if it is a 1. As each branch is traversed, we output the corresponding binary 
label on the branch. Consider, for example, the message sequence (10011). For this input 
we follow the path abcabd, and therefore output the sequence (11, 10, 11, 11, 01), which 


TABLE 10.7 State table 
for the convolutional 
encoder of Figure 10.13a 







State Binary Description 
a 00 
b 10 
c 01 
d 11 
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Ficure 10.16 (a) A portion of the central part of the trellis for the encoder of Figure 10.13a 
(b) State diagram of the convolutional encoder of Figure 10.13a. 


agrees exactly with our previous result. Thus, the input-output relation of a convolutional 
encoder is also completely described by its state diagram. 


10.6 Maximum Likelihood 
Decoding of Convolutional Codes 





Now that we understand the operation of a convolutional encoder, the next issue to be 
considered is the decoding of a convolutional code. In this section we first describe the 
underlying theory of maximum likelihood decoding, and then present an efficient algo- 
rithm for its practical implementation. 

Let m denote a message vector, and c denote the corresponding code vector applied 
by the encoder to the input of a discrete memoryless channel. Let r denote the received 
vector, which may differ from the transmitted code vector due to channel noise. Given the 
received vector r, the decoder is required to make an estimate th of the message vector. 
Since there is a one-to-one correspondence between the message vector m and the code 
vector c, the decoder may equivalently produce an estimate ĉ of the code vector. We may 
then put m = m if and only if ê = c. Otherwise, a decoding error is committed in the 
receiver. The decoding rule for choosing the estimate ¢, given the received vector r, is said 
to be optimum when the probability of decoding error is minimized. From the material 
presented in Chapter 6, we may state that for equiprobable messages, the probability of 
decoding error is minimized if the estimate ĉ is chosen to maximize the log-likelibood 
function. Let p(r|c) denote the conditional probability of receiving r, given that ¢ was sent. 
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The log-likelihood function equals log p(r|c). The maximum likelihood decoder or deci- 
sion rule is described as follows: 


Choose the estimate ¢ for which the 


log-likelihood function log p(r|c) is maximum. (10.56) 


Consider now the special case of a binary symmetric channel. In this case, both the 
transmitted code vector c and the received vector r represent binary sequences of length 
N, say. Naturally, these two sequences may differ from each other in some locations be- 
cause of errors due to channel noise. Let c; and r; denote the ith elements of c and r, 
respectively. We then have 


N 
p(t|c) = II plri|ci) (10.57) 


Correspondingly, the log-likelihood is 


N 
log p(t|c) = > log p(ri| ci) (10.58) 
Let the transition probability p(r;|c;) be defined as 
_ |p ifr; # G f 
bir:le) = p Se ieee (10.59) 


Suppose also that the received vector r differs from the transmitted code vector c in exactly 
d positions. The number d is the Hamming distance between vectors r and c. Then, we 
may rewrite the log-likelihood function in Equation (10.58) as 


log p(t|c) = d log p + (N — d) log(1 — p) 
(10.60) 





p 
= + x 
d tos = ;) N log(1 — p) 
In general, the probability of an error occurring is low enough for us to assume p < 1/2. 
We also recognize that N log(1 — p) is a constant for all c. Accordingly, we may restate 
the maximum-likelihood decoding rule for the binary symmetric channel as follows: 


Choose the estimate ĉ that minimizes the Hamming distance (10.61) 
between the received vector r and the transmitted vector c. / 


That is, for the binary symmetric channel, the maximum-likelihood decoder reduces to a 
minimum distance decoder. In such a decoder, the received vector r is compared with each 
possible transmitted code vector c, and the particular one closest to r is chosen as the 
correct transmitted code vector. The term “closest” is used in the sense of minimum num- 
ber of differing binary symbols (i.e., Hamming distance) between the code vectors under 
investigation. 


eg THE VITERBI ALGORITHM’ 


The equivalence between maximum likelihood decoding and minimum distance decoding 
for a binary symmetric channel implies that we may decode a convolutional code by choos- 
ing a path in the code tree whose coded sequence differs from the received sequence in the 
fewest number of places. Since a code tree is equivalent to a trellis, we may equally limit 
our choice to the possible paths in the trellis representation of the code. The reason for 
preferring the trellis over the tree is that the number of nodes at any level of the trellis 
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does not continue to grow as the number of incoming message bits increases; rather, it 
remains constant at 2“~1, where K is the constraint length of the code. 

Consider, for example, the trellis diagram of Figure 10.15 for a convolutional code 
with rate r = 1/2 and constraint length K = 3. We observe that at level j = 3, there are 
two paths entering any of the four nodes in the trellis. Moreover, these two paths will be 
identical onward from that point. Clearly, a minimum distance decoder may make a de- 
cision at that point as to which of those two paths to retain, without any loss of perfor- 
mance. A similar decision may be made at level j = 4, and so on. This sequence of decisions 
is exactly what the Viterbi algorithm does as it walks through the trellis. The algorithm 
operates by computing a metric or discrepancy for every possible path in the trellis. The 
metric for a particular path is defined as the Hamming distance between the coded sequence 
represented by that path and the received sequence. Thus, for each node (state) in the trellis 
of Figure 10.15 the algorithm compares the two paths entering the node. The path with 
the lower metric is retained, and the other path is discarded. This computation is repeated 
for every level j of the trellis in the range M <j < L, where M = K — 1 is the encoder’s 
memory and L is the length of the incoming message sequence. The paths that are retained 
by the algorithm are called survivor or active paths. For a convolutional code of constraint 
length K= 3, for example, no more than 2* 1 = 4 survivor paths and their metrics will 
ever be stored. This list of 2*~' paths is always guaranteed to contain the maximum- 
likelihood choice. 

A difficulty that may arise in the application of the Viterbi algorithm is the possibility 
that when the paths entering a state are compared, their metrics are found to be identical, 
In such a situation, we make the choice by flipping a fair coin (i.e., simply make a guess), 

In summary, the Viterbi algorithm is a maximum-likelihood decoder, which is op- 
timum for an AWGN channel. It proceeds in a step-by-step fashion as follows: 


Initialization 


Label the left-most state of the trellis (i.e., the all-zero state at level 0) as 0, since 
there is no discrepancy at this point in the computation. 


Computation step j + 1 
Let j = 0, 1, 2, ..., and suppose that at the previous step j we have done two things: 


» All survivor paths are identified. 
> The survivor path and its metric for each state of the trellis are stored. 


Then, at level (clock time) j + 1, compute the metric for all the paths entering each state 
of the trellis by adding the metric of the incoming branches to the metric of the connecting 
survivor path from level j. Hence, for each state, identify the path with the lowest metric 
as the survivor of step j + 1, thereby updating the computation. 


Final Step 


Continue the computation until the algorithm completes its forward search through 
the trellis and therefore reaches the termination node (i.e., all-zero state), at which time it 
makes a decision on the maximum likelihood path. Then, like a block decoder, the se- 
quence of symbols associated with that path is released to the destination as the decoded 
version of the received sequence. In this sense, it is therefore more correct to refer to the 
Viterbi algorithm as a maximum likelibood sequence estimator. 

However, when the received sequence is very long (near infinite), the storage require- 
ment of the Viterbi algorithm becomes too high, and some compromises must be made. 
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The approach usually taken is to “truncate” the path memory of the decoder as described 
here. A decoding window of length £ is specified, and the algorithm operates on a corre- 
sponding frame of the received sequence, always stopping after £ steps. A decision is then 
made on the “best” path and the symbol associated with the first branch on that path is 
released to the user. The symbol associated with the last branch of the path is dropped. 
Next, the decoding window is moved forward one time interval, and a decision on the 
next code frame is made, and so on. The decoding decisions made in this way are no longer 
truly maximum likelihood, but they can be made almost as good provided that the decod- 
ing window is long enough. Experience and analysis have shown that satisfactory results 
are obtained if the decoding window length £ is on the order of 5 times the constraint 
length K of the convolutional code or more. 


> EXAMPLE 10.6 Correct Decoding of Received All-Zero Sequence 


Suppose that the encoder of Figure 10.134 generates an all-zero sequence that is sent over a 
binary symmetric channel, and that the received sequence is (0100010000. . .). There are two 
errors in the received sequence due to noise in the channel: one in the second bit and the other 
in the sixth bit. We wish to show that this double-error pattern is correctable through the 
application of the Viterbi decoding algorithm. 

In Figure 10.17, we show the results of applying the algorithm for level j = 1, 2, 3, 4, 
5. We see that for j = 2 there are (for the first time) four paths, one for each of the four states 
of the encoder. The figure also includes the metric of each path for each level in the 
computation. , 

In the left side of Figure 10.17, for j = 3 we show the paths entering each of the states, 
together with their individual metrics. In the right side of the figure, we show the four survivors 
that result from application of the algorithm for level j = 3, 4, 5. 

Examining the four survivors in Figure 10.17 for j = 5, we see that the all-zero path 
has the smallest metric and will remain the path of smallest metric from this point forward. 
This clearly shows that the all-zero sequence is the maximum likelihood choice of the Viterbi 
decoding algorithm, which agrees exactly with the transmitted sequence. 4 


> ExaMPLE 10.7 Incorrect Decoding of Received All-Zero Sequence 


Suppose next that the received sequence is (1100010000 .. .), which contains three errors 
compared to the transmitted all-zero sequence. 

In Figure 10.18, we show the results of applying the Viterbi decoding algorithm for j = 
1, 2, 3, 4. We see that in this example the correct path has been eliminated by level j = 3. 
Clearly, a triple-error pattern is uncorrectable by the Viterbi algorithm when applied to a 
convolutional code of rate 1/2 and constraint length K = 3. The exception to this rule is a 
triple-error pattern spread over a time span longer than one constraint length, in which case 
it is very likely to be correctable. aq 


@ FREE DISTANCE OF A CONVOLUTIONAL CODE ` 


The performance of a convolutional code depends not only on the decoding algorithm 
used but also on the distance properties of the code. In this context, the most important 
single measure of a convolutional code’s ability to combat channel noise is the free distance, 
denoted by deee The free distance of a convolutional code is defined as the minimum 
Hamming distance between any two code words in the code. A convolutional code with 
free distance dree can correct ¢ errors if and only if diee is greater than 21. 

The tree distance can be obtained quite simply from the state diagram of the con- 
volutional encoder. Consider, for example, Figure 10.16), which shows the state diagram 
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FIGURE 10.17 Illustrating steps in the Viterbi algorithm for Example 10.6. 
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FIGURE 10.18 Illustrating breakdown of the Viterbi algorithm in Example 10.7. 


of the encoder of Figure 10.134. Any nonzero code sequence corresponds to a complete 
path beginning and ending at the 00 state (i.e., node a). We thus find it useful to split this 
node in the manner shown in the modified state diagram of Figure 10.19, which may be 
viewed as a signal-flow graph with a single input and a single output. A signal-flow graph 
consists of nodes and directed branches; it operates by the following rules: 


1. A branch multiplies the signal at its input node by the transmittance characterizing 
that branch. 

2. A node with incoming branches sums the signals produced by all of those branches. 

3. The signal at a node is applied equally to all the branches outgoing from that node. 

4. The transfer function of the graph is the ratio of the output signal to the input signal. 
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FIGURE 10.19 Modified state diagram of convolutional encoder. 


Returning to the signal-flow graph of Figure 10.19, we note that the exponent of D ona 
branch in this graph describes the Hamming weight of the encoder output corresponding 
to that branch. The exponent of L is always equal to one, since the length of each branch 
is one. Let T(D, L) denote the transfer function of the signal-flow graph, with D and L 
playing the role of dummy variables. For the example of Figure 10.19, we may readily use 
rules 1, 2, and 3 to obtain the following input-output relations: 


b = D’La + Le 
c= DLb + DLd 
d = DLb + DLd 
a= D?Le 

where ao, b, c, d, and a, denote the node signals of the graph. Solving the set of Equations 


(10.62) for the ratio a;/ao, we find that the transfer function of the graph in Figure 10.19 
is given by 


(10.62) 


DSL? 
TD, L) = pir (10.63) 


Using the binomial expansion, we may equivalently write 


oo 


T(D, L) = DSL? Ù (DL(1 + L)’ (10.64) 
=o 
Setting L = 1 in Equation (10.64), we thus get the distance transfer function expressed in 
the form of a power series: 


T(D, 1) = D5 + 2D6 + 4D7 +». (10.65) 


Since the free distance is the minimum Hamming distance between any two code words 
in the code and the distance transfer function T(D, 1) enumerates the number of code 
words that are a given distance apart, it follows that the exponent of the first term in the 
expansion of T(D, 1) defines the free distance. Thus, on the basis of Equation (10.65), the 
convolutional code of Figure 10.134 has a free distance diee = 5. 

This result indicates that up to two errors in the received sequence are correctable, 
for two or fewer transmission errors will cause the received sequence to be at most at a 
Hamming distance of 2 from the transmitted sequence but at least at a Hamming distance 
of 3 from any other code sequence in the code. In other words, in spite of the presence of 
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TABLE 10.8 Maximum free distances 
attainable with systematic and 
nonsystematic convolutional codes 














of rate 1/2 

Constraint 

Length K Systematic Nonsystematic 
2 3 3 
3 4 5 
4 4 6 
5 5 7 
6 6 8 
7 6 10 
8 7. 10 














any pair of transmission errors, the received sequence remains closer to the transmitted 
sequence than any other possible code sequence. However, this statement is no longer true 
if there are three or more closely spaced transmission errors in the received sequence. These 
observations confirm the results reported earlier in Examples 10.6 and 10.7. 

In using the distance transfer function T(D, 1) to calculate the free distance of a 
convolutional code, it is assumed that the power series in the unit-delay variable D rep- 
resenting T(D, 1) is convergent (i.e., its sum has a “finite” value). This assumption is 
required to justify the expansion given in Equation (10.65) for the convolutional code of 
Figure 10.134. However, there is no guarantee that T(D, 1) is always convergent. When 
T(D, 1) is nonconvergent, an infinite number of decoding errors are caused by a finite 
number of transmission errors; the convolutional code is then subject to catastrophic error 
propagation, and the code is called a catastrophic code." In this context it is noteworthy 
that a systematic convolutional code cannot be catastrophic. Unfortunately, for a pre- 
scribed constraint length K, the free distances that can be attained with systematic con- 
volutional codes using schemes such as those shown in Figure 10.13 are usually smaller 
than for the case of nonsystematic convolutional codes, as indicated in Table 10.8. 


@ ASYMPTOTIC CODING GAIN"! 


The transfer function of the encoder state diagram, modified in a manner similar to that 
illustrated in Figure 10.19, may be used to evaluate a bound on the bit error rate for a 
given decoding scheme; details of this evaluation are, however, beyond the scope of our 
present discussion. Here we simply summarize the results for two special channels, namely, 
the binary symmetric channel and the binary-input additive white Gaussian noise (AWGN) 
channel, assuming the use of binary phase-shift keying (PSK) with coherent detection. 


1. Binary symmetric channel. The binary symmetric channel may be modeled as an ad- 
ditive white Gaussian noise channel with binary phase-shift keying (PSK) as the modula- 
tion and with hard-decision demodulation. The transition probability p of the binary sym- 
metric channel is then equal to the bit error rate (BER) for the uncoded binary PSK system. 
From Chapter 6 we recall that for large values of E,/No, the ratio of signal energy per bit- 
to-noise power spectral density, the bit error rate for binary PSK without coding is dom- 
inated by the exponential factor exp(—E,/No). On the other hand, the bit error rate for 
the same modulation scheme with convolutional coding is dominated by the exponential 
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factor exp(—deerEs/2No), where 7 is the code rate and dges is the free distance of the 
convolutional code. Therefore, as a figure of merit for measuring the improvement in error 
performance made by the use of coding with hard-decision decoding, we may use the 
exponents to define the asymptotic coding gain (in decibels) as follows: 





G, = 10 logo“) dB (10.66) 
2. Binary-input AWGN channel. Consider next the case of a memoryless binary-input 
AWGN channel with no output quantization [i.e., the output amplitude lies in the interval 
(—e, œ)]. For this channel, theory shows that for large values of E,/No the bit error rate 
for binary PSK with convolutional coding is dominated by the exponential factor 
exp(—direcTE,/No), where the parameters are as previously defined. Accordingly, in this 
case, we find that the asymptotic coding gain is defined by 


Ga = 10 logio(dpeer) dB (10.67) 


From Equations (10.66) and (10.67) we see that the asymptotic coding gain for the 
binary-input AWGN channel is greater than that for the binary symmetric channel by 3 
dB. In other words, for large E,/No, the transmitter for a binary symmetric channel must 
generate an additional 3 dB of signal energy (or power) over that fora binary-input AWGN 
channel if we are to achieve the same error performance. Clearly, there is an advantage to 
be gained by permitting an unquantized demodulator output instead of making hard de- 
cisions. This improvement in performance, however, is attained at the cost of increased 
decoder complexity due to the requirement for accepting analog inputs. 

The asymptotic coding gain for a binary-input AWGN channel is approximated to 
within about 0.25 dB by a binary input Q-ary output discrete memoryless channel with 
the number of representation levels Q = 8. This means that we may avoid the need for 
an analog decoder by using a soft-decision decoder that performs finite output quantization 
(typically, Q = 8), and yet realize a performance close to the optimum. 


i 10.7 Trellis-Coded Modulation” 


In the traditional approach to channel coding described in the preceding sections of the 
chapter, encoding is performed separately from modulation in the transmitter; likewise for 
decoding and detection in the receiver. Moreover, error control is provided by transmitting 
additional redundant bits in the code, which has the effect of lowering the information bit 
rate per channel bandwidth. That is, bandwidth efficiency is traded for increased power 
efficiency. 

To attain a more effective utilization of the available bandwidth and power, coding 
and modulation have to be treated as a single entity. We may deal with this new situation 
by redefining coding as the process of imposing certain patterns on the transmitted signal. 
Indeed, this definition includes the traditional idea of parity coding. 

Trellis codes for band-limited channels result from the treatment of modulation and 
coding as a combined entity rather than as two separate operations, The combination itself 
is referred to as trellis-coded modulation (TCM). This form of signaling has three basic 
features: 


1. The number of signal points in the constellation used is larger than what is required 
for the modulation format of interest with the same data rate; the additional points 
allow redundancy for forward error-control coding without sacrificing bandwidth. 
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2. Convolutional coding is used to introduce a certain dependency between successive 
signal points, such that only certain patterns or sequences of signal points are 
permitted. 

_ 3. Soft-decision decoding is performed in the receiver, in which the permissible sequence 
of signals is modeled as a trellis structure; hence, the name “trellis codes.” 


This latter requirement is the result of using an enlarged signal constellation. By increasing 
the size of the constellation, the probability of symbol érror increases for a fixed signal- 
to-noise ratio. Hence, with hard-decision demodulation we would face a performance loss 
before we begin. Performing soft-decision decoding on the combined code and modulation 
trellis ameliorates this problem. 

In the presence of AWGN, maximum likelihood decoding of trellis codes consists of 
finding that particular path through the trellis with minimum squared Euclidean distance 
to the received sequence. Thus, in the design of trellis codes, the emphasis is on maximizing 
the Euclidean distance between code vectors (or, equivalently, code words) rather than 
maximizing the Hamming distance of an error-correcting code. The reason for this ap- 
proach is that, except for conventional coding with binary PSK and QPSK, maximizing 
the Hamming distance is not the same as maximizing the squared Euclidean distance. 
Accordingly, in what follows, the Euclidean distance is adopted as the distance measure 
of interest. Moreover, while a more general treatment is possible, the discussion is (by 
choice) confined to the case of two-dimensional constellations of signal points. The im- 
plication of such a choice is to restrict the development of trellis codes to multilevel am- 
plitude and/or phase modulation schemes such as M-ary PSK and M-ary QAM. 

The approach used to design this type of trellis codes involves partitioning an M-ary 
constellation of interest successively into 2, 4, 8, . . . subsets with size M/2, M/4, M/8,..., 
and having progressively Jarger increasing minimum Euclidean distance between their re- 
spective signal points. Such a design approach by set partitioning represents the “key idea” 
in the construction of efficient coded modulation techniques for band-limited channels. 

In Figure 10.20, we illustrate the partitioning procedure by considering a circular 
constellation that corresponds to 8-PSK. The figure depicts the constellation itself and the 
2 and 4 subsets resulting from two levels of partitioning. These subsets share the common 
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Ficure 10.20 Partitioning of 8-PSK constellation, which shows that dy < d; < da. 
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Ficure 10.21 Partitioning of 16-QAM constellation, which shows that do < dı < dz < dz. 


property that the minimum Fuclidean distances between their individual points follow an 
increasing pattern: dy < d, < d2. 

Figure 10.21 illustrates the partitioning of a rectangular constellation corresponding 
to 16-QAM. Here again we see that the subsets have increasing within-subset Euclidean 
distances: dy < dy < dz < dz. ; 

Based on the subsets resulting from successive partitioning of a two-dimensional 
constellation, we may devise relatively simple and yet highly effective coding schemes. 
Specifically, to send 7 bits/symbol with quadrature modulation (i.e., one that has in-phase 
and quadrature components), we start with a two-dimensional constellation of 2"*" signal 
points appropriate for the modulation format of interest; a circular grid is used for M-ary 
PSK, and a rectangular one for M-ary QAM. In any event, the constellation is partitioned 
into 4 or 8 subsets. One or two incoming bits per symbol enter a rate-1/2 or rate-2/3 
binary convolutional encoder, respectively; the resulting two or three coded bits per symbol 
determine the selection of a particular subset. The remaining uncoded data bits determine 
which particular point from the selected subset is to be signaled. This class of trellis codes 
is known as Ungerboeck codes. ` 

Since the modulator has memory, we may use the Viterbi algorithm to perform 
maximum likelihood sequence estimation at the receiver. Each branch in the trellis of the 
Ungerboeck code corresponds to a subset rather than an individual signal point. The first 
step in the detection is to determine the signal point within each subset that is closest to 
the received signal point in the Euclidean sense. The signal point so determined and its 
metric (i.e., the squared Euclidean distance between it and the received point) may be used 
thereafter for the branch in question, and the Viterbi algorithm may then proceed in the 
usual manner. 


© UNGERBOECK CODES FOR 8-PSK 


The scheme of Figure 10.22a depicts the simplest Ungerboeck 8-PSK code for the trans- 
mission of 2 bits/symbol. The scheme uses a rate-1/2 convolutional encoder; the corre- 
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FIGURE 10.22 (a) Four-state Ungerboeck code for 8-PSK; the mapper follows Figure 10.20. 
(b) Trellis of the code. 


sponding trellis of the code is shown in Figure 10.22b, which has four states. Note that 
the most significant bit of the incoming binary word is left uncoded. Therefore, each branch 
of the trellis may correspond to two different output values of the 8-PSK modulator or, 
equivalently, to one of the four 2-point subsets shown in Figure 10.20. The trellis of Figure 
10.22b also includes the minimum distance path. 

The scheme of Figure 10.234 depicts another Ungerboeck 8-PSK code for trans- 
mitting 2 bits/sample; it is next in the level of complexity. This second scheme uses a 
rate-2/3 convolutional encoder. Therefore, the corresponding trellis of the code has eight 
states, as shown in Figure 10.230. In this case, both bits of the incoming binary word are 
encoded, Hence, each branch of the trellis corresponds to a specific output value of the 
8-PSK modulator. The trellis of Figure 10.230 also includes the minimum distance path. 

Figures 10.22b and 10.236 also include the encoder states. In Figure 10.22, the state 
of the encoder is defined by the contents of the two-stage shift register. On the other hand, 
in Figure 10.23, it is defined by the content of the single-stage (top) shift register followed 
by that of the two-stage (bottom) shift register. 


672 CHAPTER 10 = ERROR-CONTROL CODING 


Rate-2/3 convolutional encoder 
F <a ae 
Flip-flop 




















8-PSK 
signal mapper Most significant bit 





oooo1ii1ii1 


00110011 





01010101 


0123456 7 lest 
Signal number Output 

















Encoder 
state 


000 


010 


100 


110 


001 


oll 


101 





111 


(b) 


FIGURE 10.23 (a) Eight-state Ungerboeck code for 8-PSK; the mapper follows Figure 10.20. 
(b) Trellis of the code with only some of the branches shown. 
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ASYMPTOTIC CODING GAIN 


Following the discussion in Section 10.6, we define the asymptotic coding gain of Unger- 
boeck codes as 





S) 
G, = 10 logio| = (10.68) 
die 
where dice is the free Euclidean distance of the code and d,.; is the minimum Euclidean 
distance of an uncoded modulation scheme operating with the same signal energy per bit. 
For example, by using the Ungerboeck 8-PSK code of Figure 10.22a, the signal constel- 
lation has 8 message points, and we send 2 message bits per point. Hence, uncoded trans- 
mission requires a signal constellation with 4 message points. We may therefore regard 
uncoded 4-PSK as the reference for the Ungerboeck 8-PSK code of Figure 10.22a. 

The Ungerboeck 8-PSK code of Figure 10.22a achieves an asymptotic coding gain 
of 3 dB, calculated as follows: 


1.. Each branch of the trellis in Figure 10.22b corresponds to a subset of two antipodal 
signal points. Hence, the free Euclidean distance diee of the code can be no larger 
than the Euclidean distance d, between the antipodal signal points of such a subset. 
We may therefore write 


dre = dz = 2 


where the distance d, is defined in Figure 10.24a; see also Figure 10.20. 
2. The minimum Euclidean distance of an uncoded QPSK, viewed as a reference op- 
erating with the same signal energy per bit, equals (see Figure 10.240) 


dret = V2 


Hence, as previously stated, the use of Equation (10.68) yields an asymptotic coding gain 
of 10 logio 2 = 3 dB. 

The asymptotic coding gain achievable with Ungerboeck codes increases with the 
number of states in the convolutional encoder. Table 10.9 presents the asymptotic coding 
gain (in dB) for Ungerboeck 8-PSK codes for increasing number of states, expressed with 
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FIGURE 10.24 Signal-space diagrams for calculation of asymptotic coding gain of Ungerboeck 
8-PSK code. (a) Definition of distance dz. (b) Definition of reference distance d,.. 
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TABLE 10.9 Asymptotic coding gain of Ungerboeck 8-PSK codes, 
with respect to uncoded 4-PSK 





Number of states 4 8 16 32 64 128 256 512 








Coding gain (dB) 3 3.6 4.1 4.6 4.8 5 5.4 5.7 





respect to uncoded 4-PSK. Note that improvements on the order of 6 dB require codes 
with a very large number of states. 


i 10.8 Turbo Codes” 


Traditionally, the design of good codes has been tackled by constructing codes with a great 
deal of algebraic structure, for which there are feasible decoding schemes. Such an ap- 
proach is exemplified by the linear block codes and convolutional codes discussed in pre- 
ceding sections. The difficulty with these traditional codes is that, in an effort to approach 
the theoretical limit for Shannon’s channel capacity, we need to increase the code-word 
length of a linear block code or the constraint length of a convolutional code, which, in 
turn, causes the computational complexity of a maximum likelihood decoder to increase 
exponentially. Ultimately, we reach a point where complexity of the decoder is so high 
that it becomes physically unrealizable. 

Various approaches have been proposed for the construction of powerful codes with 
large “equivalent” block lengths structured in such a way that the decoding can be split 
into a number of manageable steps. Building on these previous approaches, the develop- 
_ment of turbo codes and low-density parity-check codes has been by far most successful. 
Indeed, this development has opened a brand new and exciting way of constructing good 
codes and decoding them with feasible complexity. Turbo codes are discussed in this sec- 
tion and low-density parity-check codes are discussed in Section 10.10. 


TurBo CODING 


In its most basic form, the encoder of a turbo code consists of two constituent systematic 
encoders joined together by means of an interleaver, as illustrated in Figure 10.25. 

An interleaver is an input-output mapping device that permutes the ordering of a 
sequence of symbols from a fixed alphabet in a completely deterministic manner; that is, 
it takes the symbols at the input and produces identical symbols at the output but in a 
different temporal order. The interleaver can be of many types, of which the periodic and 
pseudo-random are two. Turbo codes use a pseudo-random interleaver, which operates 
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Ficure 10.25 Block diagram of turbo encoder. 
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only on the systematic bits. There are two reasons for the use of an interleaver in a turbo 
code: 


> To tie together errors that are easily made in one half of the turbo code to errors 
that are exceptionally unlikely to occur in the other half. This is indeed the main 
reason why the turbo code performs better than a traditional code. 

® To provide robust performance with respect to mismatched decoding, which is a 
problem that arises when the channel statistics are not known or have been incor- 
rectly specified. 


Typically, but not necessarily, the same code is used for both constituent encoders 
in Figure 10,25. The constituent codes recommended for turbo codes are short constraint- 
length recursive systematic convolutional (RSC) codes. The reason for making the con- 
volutional codes recursive (i.e., feeding one or more of the tap outputs in the shift register 
back to the input) is to make the internal state of the shift register depend on past outputs. 
This affects the behavior of the error patterns (a single error in the systematic bits produces 
an infinite number of parity errors), with the result that a better performance of the overall 
coding strategy is attained. 


® EXAMPLE 10.8 Eight-state RSC Encoder 


Figure 10.26 shows an example eight-state RSC encoder. The generator matrix for this re- 
cursive convolutional code is 


+D +D? a 
1+D a (10.69) 


Ba pee 


where D is the delay variable. The second entry of the matrix g(D) is the transfer function of 
the feedback shift register, defined as the transform of the output divided by the transform 
of the input. Let M(D) denote the transform of the message sequence (m,}*., and B(D) denote 
the transform of the parity sequence {b,}#-+. By definition, we have 


BID) 1+D+D?+ D> 


(D) 1+D+D° 
‘Cross-multiplying, we get: 
(1+ D + D? + D’)M(D) = (1 + D + D°)B(D) 
which, on inversion into the time domain, yields 
mi + m1 + Mi- + m3 + b; + bj. + 6-3 =0 (10.70) 
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FıGURE 10.26 Example eight-state recursive systematic convolutional (RSC) encoder. 
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where the addition is modulo-2. Equation (10.70) is the parity-check equation, which the 
convolutional encoder of Figure 10.26 satisfies at each time step $. 4 


In Figure 10.25 the input data stream is applied directly to encoder 1, and the pseudo- 
randomly reordered version of the same data stream is applied to encoder 2. The systematic 
bits (i.e., original message bits) and the two sets of parity-check bits generated by the two 
encoders constitute the output of the turbo encoder. Although the constituent codes are 
convolutional, in reality turbo codes are block codes with the block size being determined 
by the size of the interleaver. Moreover, since both RSC encoders in Figure 10.25 are 
linear, we may describe turbo codes as linear block codes. 

The block nature of the turbo code raises a practical issue: How do we know the 
beginning and the end of a code word? The common practice is to initialize the encoder 
to the all-zero state and then encode the data. After encoding a certain number of data 
bits a number of tail bits are added so as to make the encoder return to the all-zero state 
at the end of each block; thereafter the cycle is repeated. The termination approaches of 
turbo codes include the following: 


» A simple approach is to terminate the first RSC code in the encoder and leave the 
second one unterminated. A drawback of this approach is that the bits at the end of 
the block due to the second RSC code are more vulnerable to noise than the other 
bits. Experimental work has shown that turbo codes exhibit a leveling off in perfor- 
mance as the SNR increases. This behavior is not like an error floor, but it has the 
appearance of an error floor compared to the steep drop in error performance at low 
SNR. This error floor is affected by a number of factors, the dominant one of which 
is the choice of interleaver. 

> A more refined approach" is to terminate both constituent codes in the encoder in 
a symmetric manner. Through the combined use of a good interleaver and dual 
termination, the error floor can be reduced by an order of magnitude compared to 
the simple termination approach. 


In the original version of the turbo encoder, the parity-check bits generated by the 
two encoders in Figure 10.25 were punctured prior to data transmission over the channel 
to maintain the rate at 1/2. A punctured code is constructed by deleting certain parity 
check bits, thereby increasing the data rate. Puncturing is the inverse of extending a code. 
It should, however, be emphasized that the use of a puncture map is not a necessary 
requirement for the generation of turbo codes. 

The novelty of the parallel encoding scheme of Figure 10.25 is in the use of recursive 
systematic convolutional (RSC) codes and the introduction of a pseudo-random interleaver 
between the two encoders. Thus a turbo code appears essentially random to the channel 
by virtue of the pseudo-random interleaver, yet it possesses sufficient structure for the 
decoding to be physically realizable. Coding theory asserts that a code chosen at random 
is capable of approaching Shannon’s channel capacity, provided that the block size is 
sufficiently large.’ This is indeed the reason behind the impressive performance of turbo 
codes, as discussed next. 


= PERFORMANCE OF TURBO CODES 





Figure 10.27 shows the error performance of a 1/2 rate, turbo code with a large block size 
for binary data transmission over an AWGN channel.!® The code uses an interleaver of 
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FIGURE 10.27 Noise performances of 1/2 rate, turbo code and uncoded transmission for 


AWGN channel; the figure also includes Shannon’s theoretical limit on channel capacity for code 
rate r= 1/2. 


size 65,536 and a BCJR-based decoder; details of this decoder are presented later in the 
section. Eighteen iterations of turbo decoding were used in the computation. 

For the purpose of comparison, Figure 10.27 also includes two other curves for the 
same AWGN channel: 


 Uncoded transmission (i.e., code rate r = 1). 
» Shannon’s theoretical limit for code rate 1/2, which follows from Figure 9.18b. 


From Figure 10.27, we may draw two important conclusions: 


1. Although the bit error rate for the turbo-coded transmission is significantly higher 
than that for uncoded transmission at low E;/No, the bit error rate for the turbo- 
coded transmission drops very rapidly once a critical value of E,/No has been 
reached. 

2. At a bit error rate of 1075, the turbo code is less than 0.5 dB from Shannon’s theo- 
retical limit. 


Note, however, attaining this highly impressive performance requires that the size of 
the interleaver, or, equivalently, the block length of the turbo code, be large. Also, the 
large number of iterations needed to improve performance increases the decoder latency. 
This drawback is due to the fact that the digital processing of information does not lend 
itself readily to the application of feedback, which is a distinctive feature of the turbo 
decoder. 

Now that we have an appreciation for the impressive performance of turbo codes, 
the stage is set for a discussion of how turbo decoding is actually performed. 


a TURBO DECODING 


Turbo codes derive their distinctive name from analogy of the decoding algorithm to the 
“turbo engine” principle. Figure 10.284 shows the basic structure of the turbo decoder. It 
operates on noisy versions of the systematic bits and the two sets of parity-check bits in 
two decoding stages to produce an estimate of the original message bits. 
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FIGURE 10.28 (a) Block diagram of turbo decoder. (b) Extrinsic form of turbo decoder, where I 
stands for interleaver, D for de-interleaver, and BCJR for BCJR algorithm for log-MAP decoding. 


Each of the two decoding stages uses a BCJR algorithm,” which was originally 


invented by Bahl, Cocke, Jelinek, and Raviv (hence the name) to solve a maximum a 
posteriori probability (MAP) detection problem. The BCJR algorithm differs from the 
Viterbi algorithm in two fundamental respects: 


1. 


The BCJR algorithm is a soft input-soft output decoding algorithm with two recur- 
sions, one forward and the other backward, both of which involve soft decisions. In 
contrast, the Viterbi algorithm is a soft input—hard output decoding algorithm, with 
a single forward recursion involving soft decisions; the recursion ends with a hard 
decision, whereby a particular survivor path among several ones is retained, In com- 
putational terms, the BCJR algorithm is therefore more complex than the Viterbi 
algorithm because of the backward recursion. 

The BCJR algorithm is a MAP decoder in that it minimizes the bit errors by esti- 
mating the a posteriori probabilities of the individual bits in a code word; to recon- 
struct the original data sequence, the soft outputs of the BCJR algorithm are hard- 
limited. On the other hand, the Viterbi algorithm is a maximum likelihood sequence 
estimator in that it maximizes the likelihood function for the whole sequence, not 
each bit. As such, the average bit error rate of the BCJR algorithm can be slightly 
better than the Viterbi algorithm; it is never worse. 


Most important, formulation of the BCJR algorithm rests on the fundamental assumptions 


that 


(1) the channel encoding, namely, the convolutional encoding performed in the trans- 


mitter, is modeled as a Markov process, and (2) the channel is memoryless. In the context 
of our present discussion, the Markovian assumption means that if a code can be repre- 
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sented as a trellis, then the present state of the trellis depends only on the past state and 
the input bit. (A mathematical treatment of the BCJR algorithm is given later in this 
section.) 

Before proceeding to describe the operation of the two-stage turbo decoder in Figure 
10.284, we find it desirable to introduce the notion of extrinsic information. The most 
convenient representation for this concept is as a log-likelihood ratio, in which case ex- 
trinsic information is computed as the difference between two log-likelihood ratios as 
depicted in Figure 10.29. Formally, extrinsic information, generated by a decoding stage 
for a set of systematic (message) bits, is defined as the difference between the log-likelihood 
ratio computed at the output of that decoding stage and the intrinsic information repre- 
sented by a log-likelihood ratio fed back to the input of the decoding stage. In effect, 
extrinsic information is the incremental information gained by exploiting the dependencies 
that exist between a message bit of interest and incoming raw data bits processed by the 
decoder. 

On this basis, we may depict the flow of information in the two-stage turbo decoder 
of Figure 10.284 in a symmetric extrinsic manner as shown in Figure 10.28). The first 
decoding stage uses the BCJR algorithm to produce a soft estimate of systematic bit x;, 
expressed as the log-likelihood ratio 

Lix) = los He Au, by 2e), j=1,2,...,k (10.71) 
P(x; = Olu, bi L,(x)) 


where u is the set of noisy systematic bits, ¢, is the set of noisy parity-check bits generated 


by encoder 1, and /,(x) is the extrinsic information about the set of message bits x derived 
from the second decoding stage and fed back to the first stage. Assuming that the & message 
bits are statistically independent, the total log-likelihood ratio at the output of the first 
decoding stage is therefore 


k 
L(x) = X hix) (10.72) 
j=1 


Hence, the extrinsic information about the message bits derived from the first decoding 
stage is f 


iy(x) = h(x) - hx) (10.73) 


where /,(x) is to be defined. 


Before application to the second decoding stage, the extrinsic information |4 (x) is re- 
ordered to compensate for the psuedo-random interleaving introduced in the turbo en- 
coder. In addition, the noisy parity-check bits €, generated by encoder 2 are used as input. 
Thus by using the BCJR algorithm, the second decoding stage produces a more refined 
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FIGURE 10.29 Illustrating the concept of extrinsic information. 
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soft estimate of the message bits x. This estimate is re-interleaved to produce the total log. 


likelihood ratio [,(x). The extrinsic information L(x) fed back to the first decoding stage 
is therefore 


Lix) = bx- f(x) (10.74) 


where i,(x) is itself defined by Equation (10.73), and h(x) is the log-likelihood ratio com- 
puted by the second stage. Specifically, for the jth element of the vector x, we have 


P(x; = 1|u, bo, U(x) 
P(x; = Olu, bz h(x) 





h(x) = oes ), j=1,2,...,2 (10.75) 


Through the application of /,(x) to the first stage, the feedback loop around the pair of 
decoding stages is thereby closed. Note that although in actual fact the set of noisy sys- 
tematic bits u is only applied to the first decoding stage as in Figure 10.284, by formulating 
the information flow in the symmetric extrinsic manner depicted in Figure 10.286 we find 
that u is, in effect, also applied to the second decoding stage. aa 

An estimate of the message bits x is computed by hard-limiting the log-likelihood 
ratio [,(x) at the output of the second stage, as shown by 


& = sgn(h(x)) (10.76) 


where the signum function operates on each element of h(x) individually. 

To initiate the turbo decoding algorithm, we simply set h(x) = 0 on the first itera- 
tion of the algorithm; see Figure 10.286. 

The motivation for feeding only extrinsic information from one stage to the next in 
the turbo decoder of Figure 10.28 is to maintain as much statistical independence between 
the bits as possible from one iteration to the next. The feedback decoding strategy described 
herein implicitly relies on this assumption. If this assumption of statistical independence 
is strictly true, it can be shown that the estimate £ defined in Equation (10.76) approaches 
the MAP solution as the number of iterations approaches infinity.'* The assumption of 
statistical independence appears to be close to the truth in the vast majority of cases en- 
countered in practice. 


u THE BCJR ALGORITHM 


For a discussion of turbo decoding to be complete, a mathematical exposition of the BCJR 
algorithm for MAP estimation is in order. 

Let x(t) be the input to a trellis encoder at time t. Let y(t) be the corresponding output 
observed at the receiver. Note that y(t) may include more than one observation; for ex- 
ample, a rate 1/n code produces » bits for each input bit, in which case we have an 
n-dimensional observation vector. Let the observation vector be denoted by 


Yaz) = [y(4), y(2), sarees, y(t] 


Let A,,(t) denote the probability that a state s(t) of the trellis encoder equals m, where 
m= 1,2,...,M. We may then write 





A(t) = PEs(t) |y] (10.77) 
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where s(t) and A(t) are both M-by-1 vectors. Then, for a rate 1/7 linear convolutional code 
with feedback as in the RSC code, the probability that a symbol “1” was the message bit 
is given by 


x(t) =1ly)= 5 ale (10.78) 


SEF 4 


where F, is the set of transitions that correspond to a symbol “1” at the input, and A,(2) 
is the s-component of A(t). 
Define the forward estimation of state pigbabilities as the M-by-1 vector 


a(t) = P(s(t)| ya.) (10.79) 


where the observation vector Yq,» is defined above. Also define the backward estimation 
of state probabilities as the M-by-1 vector 


B(t) = P(s(t)| Yee) (10.80) 
where 
Vina) = [y(2), y(t + 1), Ce) y(k)] 


The vectors a(t) and B(t) are estimates of the state probabilities at time ż based on the past 
and future data, respectively. We may then formulate the separability theorem as follows: 


The state probabilities at time ¢ are related to the forward estimator a(t) and back- 

ward estimator B(z) by the vector 

ote) BO 
|| x(t) © Blt) Ya 


where a(t) + B(z) is the vector product of a(t) and B(t), and || a(t) + BZ) ||, is the 
L, norm of this vector product. 


A(t) = (10.81) 


The vector product a(t) B(t) (not to be confused with the inner product) is defined in 
terms of the individual elements of a(t) and B(z) by 


a, (t)B,(t) 
aipaga] nen (10.82) 
au(t)Bu(t) 
and the L, norm of a(t) - B(z) is defined by 
|| æt) © Biz) Ila = > Cyt) Brt) (10.83) 


The separability theorem says that the state distribution at time ż given the past is 
independent of the state distribution at time ¢ given the future, which is intuitively satisfying 
recalling the Markovian assumption for channel encoding, which is basic to the BCJR 
algorithm. Moreover, this theorem provides the basis of a simple way of combining the 
forward and backward estimates to obtain a complete description of the state probabilities. 

To proceed further, let the state transition probability at time ż be defined by 


Ymmlt) = P(s(t) = m, y(t)|s(t — 1) = m’) (10.84) 
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and denote the M-by-M matrix of transition probabilities as 
TE) = Yeyn(t) (10.85) 


We may then formulate the recursion theorem as follows: 


The forward estimate a(t) and backward estimate B(t) are computed recursively as 


a(t — YT) 


Tip = —— 
e © = Tate roh 


(10.86) 


and 


T(t + 1)pit + 1) 
|| Te + pe + 1) lh 





B(t) = (10.87) 


where the superscript T denotes matrix transposition. 


The separability and recursion theorems together define the BCJR algorithm for the 
computation of a posteriori probabilities of the states and transitions of a code trellis, 
given the observation vector. Using these estimates, the likelihood ratios needed for turbo 
decoding may then be computed by performing summations over selected subsets of states 
as required. 


10.9 Computer Experiment: 
Turbo Decoding 


Two properties constitute the hallmark of turbo codes: 


Property 1: 


The error performance of the turbo decoder improves with the number of iterations of the 
decoding algorithm. This is achieved by feeding extrinsic information from the output of 
the first decoding stage to the input of the second decoding stage in the forward path and 
feeding extrinsic information from the output of the second stage to the input of the first 
stage in the backward path, and then permitting the iterative decoding process to take its 
natural course in response to the received noisy message and parity bits. 


Property 2 

The turbo decoder is capable of approaching the Shannon theoretical limit of channel 
capacity in a computationally feasible manner; this property has been demonstrated ex- 
perimentally but not yet proven theoretically. 


Property 2 requires that the block length of the turbo code be large. Unfortunately, a 
demonstration of this property requires the use of sophisticated implementations of the 
turbo decoding algorithm that are beyond the scope of this book. Accordingly, we focus 
our attention on a demonstration of Property 1 in this computer experiment. 

So, as the primary objective of this computer experiment, we wish to use the log- 
MAP implementation of the BCJR algorithm to demonstrate Property 1 of turbo decoding. 
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FIGURE 10.30 Results of the computer experiment on turbo decoding, for increasing number of 
iterations. 


The only channel impairment assumed in the experiment is additive white Gaussian noise. 
Details of the turbo encoder and decoder are as follows: 
Turbo Encoder (described in Figure 10.25): 


Encoder 1: convolutional encoder [1, 1, 1] 
Encoder 2: convolutional encoder [1, 0, 1] 
Block (i.e., interleaver) length: 1,200 bits 


Turbo Decoder (described in Figure 10.28): 
The BCJR algorithm for log-MAP decoding. 


The experiment was carried out for E,/No = 1, 1.5, 2, and 2.5 dB, with varying 
number of iterations at each E,/No. For each trial of the experiment, the number of bit 
errors was calculated after accumulating a total of 20 blocks of data (each 1,200 bits long) 
that were noise-corrupted. The probability of error was then evaluated as the ratio of bit 
errors to the total number of encoded bits. Note that in this calculation, many of the blocks 
of encoded bits were correctly decoded. 

The results of the experiment are plotted in Figure 10.30. The following observations 
can be made from this figure: 


1. For a given E,/No, the probability of error decreases with increasing number of 
iterations, confirming Property 1 of turbo decoding. 
2. After eight iterations, there is no significant improvement in decoding performance. 


3. For a fixed number of iterations, the probability of error decreases with increasing 
E,/No; which is to be expected. 


10.10 Low-Density Parity-Check Codes? 
y ty 


Turbo codes, discussed in Section 10.8, and low-density parity-check (LDPC) codes, dis- 
cussed in this section, belong to a broad family of error-control coding techniques called 
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compound codes. The two most important advantages of LDPC codes over turbo codes 
are: 


© Absence of low-weight code words. 
> Iterative decoding of lower complexity. 


With regard to the issue of low-weight code words, we usually find that a small 
number of code words in a turbo code are undesirably close to the given code word. Due 
to this closeness in weights, once in a while the channel noise causes the transmitted code 
word to be mistaken for a nearby code word. Indeed, it is this behavior that is responsible 
for the error floor (typically around a bit error rate of 107° to 1076) that was mentioned 
earlier. In contrast, LDPC codes can be easily constructed so that they do not have such 
low-weight code words, and they can therefore achieve vanishingly small bit error rates, 
The error-floor problem in turbo codes can be alleviated by careful design of the 
interleaver. 

Turning next to the issue of decoding complexity, we note that the computational 
complexity of a turbo decoder is dominated by the BCJR algorithm, which operates on 
the trellis for the convolutional code used in the encoder. The number of computations in 
each recursion of the BCJR algorithm scales linearly with the number of states in the trellis, 
Commonly used turbo codes employ trellises with 16 states or more. In contrast, LDPC 
codes use a simple parity-check trellis that has just two states. Consequently, the decoders 
for LDPC codes are significantly simpler than those for turbo decoders. Moreover, being 
parallelizable, LDPC decoding may be performed at greater speeds than turbo decoding. 

However, a practical objection to the use of LDPC codes is that for large block 
lengths, their encoding complexity is high compared to turbo codes. 


= CONSTRUCTION OF LDPC CODES 


LDPC codes are specified by a parity-check matrix denoted by A, which is sparse; that is, 
it consists mainly of 0s and a small number of 1s. In particular, we speak of (n, te, t) 
LDPC codes, where 7 denotes the block length, ¢, denotes the weight (i.e., number of 1s) 
in each column of the matrix A, and t, denotes the weight of each row with ż, > t,. The 
rate of such a LDPC code is 

te 


=1->7 10.88 
r A ( ) 


whose validity may be justified as follows. Let p denote the density of 1s in the parity- 
check matrix A. Then, following the terminology introduced in Section 10.3, we may set 


t. = p(n — k) 
and 
t, = pn 


where (n — k) is the number of rows in A and 1 is the number of columns (i.e., the block 
length). Therefore, dividing ¢, by t,, we get 
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By definition, the code rate of a block code is k/n, hence the result of Equation (10.88) 
follows. For this result to hold, however, the rows of A must be linearly independent. 

The structure of LDPC codes is well portrayed by bipartite graphs. Figure 10.31 
shows such a graph for the example code of n = 10, t, = 3, and t, = 5. The left-hand 
nodes in the graph of Figure 10.31 are variable nodes, which correspond to elements of 
the code word. The right-hand nodes of the graph are check nodes, which correspond to 
the set of parity-check constraints satisfied by code words in the code. LDPC codes of the 
type exemplified by the graph of Figure 10.31 are said to be regular in that all the nodes 
of a similar kind have exactly the same degree. In the example graph of Figure 10.31, the 
degree of the variable nodes is t, = 3, and the degree of the check nodes is t, = 5. As the 
block length n approaches infinity, each check node is connected to a vanishingly small 
fraction of variable nodes, hence the term low-density. 

The matrix A is constructed by putting 1s in A at random, subject to the regularity 
constraints: 


» Each column contains a small fixed number, t., of 1s. 
» Each row contains a small fixed number, t,, of 1s. 


In practice, these regularity constraints are often violated slightly in order to avoid having 
linearly dependent rows in the parity-check matrix A. 

Unlike the linear block codes discussed in Section 10.3, the parity-check matrix A of 
LDPC codes is not systematic (i.e., it does not have the parity-check bits appearing in 
diagonal form), hence the use of a symbol different from that used in Section 10.3. Nev- 
ertheless, for coding purposes, we may derive a generator matrix G for LDPC codes by 
means of Gaussian elimination performed in modulo-2 arithmetic; this procedure is illus- 
trated later in Example 10.9. Following the terminology introduced in Section 10.3, the 
1-by-n code vector c is first partitioned as 


c= [b: m] 





Variable 
nodes 


FIGURE 10.31 Bipartite graph of the (10, 3, 5) LDPC cade. 
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where m is the k-by-1 message vector, and b is the (#-k)-by-1 parity vector; see Equation 
(10.9). Correspondingly, the parity-check matrix A is partitioned as 


A, 
AT = |..... (10.89) 


where A, is a square matrix of dimensions (n-k) X (#-k), and A; is a rectangular matrix 
of dimensions k X (n-k); transposition symbolized by the superscript T is used in the 
partitioning of matrix A for convenience of presentation. Imposing the constraint of Equa- 
tion (10.16) on the LDPC code, we may write 


A; 
[b : m]] ..... =0 
Az 
or, equivalently, 
bA, + mA, = 0 (10.90) 


Recall from Equation (10.7) that the vectors m and b are related by 
b = mP 


where P is the coefficient matrix. Hence, substituting this relation into Equation (10.90), 
we readily find that, for any nonzero message vector m, the coefficient matrix of LDPC 
codes satisfies the condition 


PA, + A, = 0 


which holds for all nonzero message vectors and, in particular, for m in the form 
[0-++010-+-+ 0] that will isolate individual rows of the generator matrix. 
Solving this equation for matrix P, we get 


P = A,A;! (10.91) 


where Aj? is the inverse of matrix Ay, which is naturally defined in modulo-2 arithmetic. 
Finally, the generator matrix of LDPC codes is defined by 


G = [P: 1] 


10.52 
= [AAT : I] nee 


where I, is the k-by-k identity matrix; see Equation (10.12). 

It is important to note that if we take the parity-check matrix A for some arbitrary 
LDPC code and just pick (7 — k} columns of A at random to form a square matrix Aj, 
there is no guarantee that A, will be nonsingular (i.e., the inverse Ay’ will exist), even if 
the rows of A are linearly independent. In fact, for a typical LDPC code with large block 
length n, such a randomly selected A, is highly unlikely to be nonsingular, because it is 
very likely that at least one row of A, will be all Os. Of course, when the rows of A are 
linearly independent, there will be some set of (# — k) columns of A that will make a 
nonsingular Ay, as illustrated in Example 10.9. For some construction methods for LDPC 
codes the first (7 — k) columns of A may be guaranteed to produce a nonsingular Aj, of 
at least do so with a high probability, but that is not true in general. 
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& ExampLe 10.9 (10, 3, 5) LDPC Code 


Consider the bipartite graph of Figure 10.31 pertaining to a (10, 3, 5) LDPC code. The parity- 
check matrix of the code is defined by 


110101:001°0 
0110101100 
1000110011 
A=ļ|011101i1000 
1010100101 
0001001111 


ye 1m 
AT AZ 
which appears to be random, while maintaining the regularity constraints: t. = 3 and ft, = 5. 
Partitioning the matrix A in the manner described in Equation (10.89): 


101010 
110100 
010110 

A, = 
100101 
011010 
101100 
010101 
010011 

B=)i 01001 
001011 


To derive the inverse of matrix A,, we first use Equation (10.90) to write 


101010 
11043100 
010110 
[bo bi, 62, 63, bas b1 0 O 1 O 1] = [ko t, H2, 43, Hay Us 
b 01410410 u = mA, 
1014100 
Ai 


where we have introduced the vector u to denote the matrix product mA). By using Gaussian 
elimination, the matrix A, is transformed into lower diagonal form (i.e., all the elements above 
the main diagonal are zero), as shown by 


A, > 


Be OF Oe 
omore oo 
O Orere Oo Oo 
een oooo 
ornooooe 
rPoooo So 


688 CHAPTER 10 = ERROR-CONTROL CODING 


This transformation is achieved by the following modulo-2 additions performed on the col- 
umns of square matrix Ay: 


Columns 1 and 2 are added to column 3. 
Column 2 is added to column 4. 

Columns 1 and 4 are added to column 5. 
Columns 1, 2 and 5 are added to column 6. 


vVrYVY 


Correspondingly, the vector u is transformed as 
u — [uo 41, Ho + t + Ha, U1 + U3, Ug + U3 + ta, Uo + Uy + Hg + tts] 


Accordingly, premultiplying the transformed matrix A, by the parity vector b, using successive 
eliminations in modulo-2 arithmetic working backwards, and putting the solutions for the 
elements of the parity vector b in terms of the elements of the vector u in matrix form, we get 


0010141 
104310041 
1131000 
[toy 415 2, 43, Ha, us| 1 1 0 0 1 0} = [bo, bi, bo, b3, b4, bs] 
Daea a rE. 
u 0 10011 b 
1 1110 1 
e 
A;! 
The inverse of matrix A, is therefore 
001011 
10310041 
111000 
Aj! = 
g 110010 
01004141 
1 1 1 10 1 


The matrix product A,A;’ is (using the given value of A, and the value of A;’ just found) 


100110 
Pee ee ee 
001110 
010110 


Finally, using Equation (10.92), the generator of the (10, 3, 5) LDPC code is 





10011 0:10 0 (0 

0 00111:0100 

G=|/0 0- L- 1- 1 070011 0 

0 1 OT i 0:0 0 0 1, 
AA? Ik 


It is important to recognize that the LDPC code described in this example is intended 
only for the purpose of illustrating the procedure involved in the generation of such a code. 
In practice, the block length » is orders of magnitude larger than that considered in this 
example. Moreover, in constructing the matrix A, we may constrain all pairs of columns to 
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have a matrix overlap (i.e., inner product of any two columns in matrix A) not to exceed 1; 
such a constraint, over and above the regularity constraints, is expected to improve the per- 
formance of LDPC codes. Unfortunately, with a small block length as that considered in this 
example, it is difficult to satisfy this additional requirement. 4 


= MINIMUM DISTANCE OF LDPC CoDES 


In practice, the block length of a LDPC code is large, ranging from 10° to 10°, which 
means that the number of code words in a particular code is correspondingly large. Con- 
sequently, the algebraic analysis of LDPC codes is rather difficult. It is much more pro- 
ductive to perform a statistical analysis on an ensemble of LDPC codes. Such an analysis 
permits us to make statistical statements about certain properties of member codes in the 
ensemble. Moreover, an LDPC code with these properties can be found with high prob- 
ability by a random selection from the ensemble. 

Among these properties, the minimum distance of the member codes is of particular 
interest. From Section 10.3 we recall that the minimum distance of a linear block code is, 
by definition, the smallest Hamming distance between any pair of code vectors in the code. 
Over an ensemble of LDPC codes, the minimum distance of a member code is naturally a 
random variable. Elsewhere”? it is shown that as the block length increases, for fixed 
t, = 3 and t, > t. the probability distribution of the minimum distance can be overbounded 
by a function that approaches a unit step function at a fixed fraction A,, of the block 
length #. Thus, for large n, practically all the LDPC codes in the ensemble have a minimum 
distance of at least n A,,. Table 10.10 presents the rate r and A, .z, of LDPC codes for 
different values of the weight-pair (t,, t,). From this table we see that for t, = 3 and t, = 6 
the code rate 7 attains its highest value of 1/2 and the fraction A,,,, attains its smallest value, 
hence the preferred choice of t, = 3 and ż, = 6 in the design of LDPC codes. 


% PROBABILISTIC DECODING OF LDPC CODES 


At the transmitter, a message vector m is encoded into a code vector c = mG, where G is 

the generator matrix for a specified weight-pair (t,, t,) and therefore minimum distance 

din» The vector c is transmitted over a noisy channel to produce the received vector 
r=cte 


where e is the error vector due to channel noise; see Equation (10.17). By construction, 
the matrix A is a parity matrix of the LDPC code; that is, AG’ = 0. Given the received 


| TABLE 10.10° The raie r and 
fractional term A, , of LDPC codes 
for varying weights t. and t, 


Rater Are. 








te t 

5 6 0.167 0.255 
4 5 0.2 0.210 
3 4 0.25 0.122 
4 6 0.333 0.129 
3 5 0.4 0.044 
3 6 0.5 0.023 








“Adapted from Gallager (1962) with permission of the IEEE. 
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vector r, the bit-by-bit decoding problem is to find the most probable vector ¢ that satisfies 
the condition ĉAT = 0. 

In what follows, a bit refers to an element of the received vector r, and a check refers 
to a row of matrix A. Let (i) denote the set of bits that participate in check i. Let $(;) 
denote the set of checks in which bit j participates. A set $(#) that excludes bit j is denoted 
by $()\j. Likewise, a set §(/) that excludes check i is denoted by #(/)\i. 

The decoding algorithm has two alternating steps: horizontal step and vertical step, 
which run along the rows and columns of matrix A, respectively. In the course of these 
steps, two probabilistic quantities associated with nonzero elements of matrix A are alter- 
nately updated. One quantity, denoted by P%, defines the probability that bit j is symbol 
x (i.e., symbol 0 or 1), given the information derived via checks performed in the horizontal 
step, except for check i. The second quantity, denoted by Q}, defines the probability that 
check i is satisfied, given that bit j is fixed at the value x and the other bits -have the 
probabilities P; : j’ E $(#)\y. 

The LDPC decoding algorithm then proceeds as follows:”* 


Initialization 
: The variables P? and P} are set equal to the a priori probabilities p? and p} of symbols 
0 and 1, respectively, with p? + p} = 1. 


Horizontal Step 
In the horizontal step of the algorithm, we run through the checks i. Define 


AP, = Pi — P} 
For each weight-pair (i, j), compute 
AQ; = II AP iy 
FESDY 
Hence, set 
ol 
Q; = > (1 + AQ;) 
oe 
Q; = 2 (1 - AQ) 


Vertical Step 


In the vertical step of the algorithm, the values of the probabilities P} and P} are 
updated using the quantities computed in the horizontal step. In particular, for each bit j, 


compute 
o o o 
Py = a; I] OF; 
#Es(jM 
pas 1 1 
Pi = ajb; I] Qr; 
rEsGN 


where the scaling factor œ; is chosen to make 


P; + P} = 
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In the vertical step, we may also update the pseudo-posterior probabilities: 
P? = ap? IT OF 
7E 447) 
ap} IT o 


ies{j) 


P; 


where a; is chosen to make 
Po + PP =1 


The quantities obtained in the vertical step are used to compute a tentative estimate 
ê. If the condition ĉAT = 0 is satisfied, the decoding algorithm is terminated. Otherwise, 
the algorithm goes back to the horizontal step. If after some maximum number of iterations 
(e.g., 100 or 200) there is no valid decoding, a decoding failure is declared. The decoding 
procedure described herein is a special case of the general low-complexity suen-product 
algorithm. 

Simply stated, the sum-product algorithm passes probabilistic quantities between the 
check nodes and variable nodes of the bipartite graph. By virtue of the fact that each 
parity-check constraint can be represented by a simple convolutional coder with one bit 
of memory, we find that LDPC decoders are simpler to implement than turbo decoders, 
as stated earlier. 

In terms of performance, however, we may say the following in light of experimental 
results reported in the literature: Regular LDPC codes do not appear to come as close to 
Shannon’s limit as do their turbo code counterparts. 


| 10.11 Irregular Codes 
The turbo codes discussed in Section 10.8 and the LDPC codes discussed in Section 10.10 
are both regular codes, each in its own individual way. The error-correcting performance 
of both of these codes over a noisy channel can be improved substantially by using their 
respective irregular forms. 

In a standard turbo code with its encoder as shown in Figure 10.25, the interleaver 
maps each systematic bit to a unique input bit of convolutional encoder 2. In contrast, 
irregular turbo codes™ use a special design of interleaver that maps some systematic bits 
to multiple input bits of the convolutional encoder. For example, each of 10 percent of 
the systematic bits may be mapped to eight inputs of the convolutional encoder instead of 


Systematic 
bits x 


Irregular Encoder Parity-check 
interleaver 1 1 bits z, 


























Message 


bits Output 

















Irregular Encoder Parity-check 
interleaver 2 a 2 3 bits z, 


Figure 10.32 Block diagram of irregular turbo encoder. 
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a single one, As shown in Figure 10.32, similar irregular interleavers are used in both 
convolutional encoding paths to generate the parity-check bits z, and z in response to the 
message bits x. Irregular turbo codes are decoded in a similar fashion to regular turbo 
codes. 

To construct an irregular LDPC code,” the degrees of the variable and check nodes 
in the bipartite graph are chosen according to some distribution. For example, we may 
have an irregular LDPC code with the following graphical representation: 


e One half of the variable nodes have degree 5 and the other half of the variable nodes 
have degree 3. 


® One half of the check nodes have degree 6 and the other half of the check nodes 
have degree 8. 


For a given block length and a given degree sequence, we define an ensemble of codes by 
choosing the edges (i.e., the connections between the variable and check nodes) in a ran- 
dom fashion. Specifically, the edges emanating from the variable nodes are enumerated in 
some arbitrary order, and likewise for the edges emanating from the check nodes. 

Figure 10.33 plots the error performances of the following codes:** 


® Irregular LDPC code: k = 50,000, 2 = 100,000, rate = 1/2 
> Turbo code (regular): k = 65,536, n = 131,072, and rate = 1/2 
> Irregular turbo code: k = 65,536, n = 131,072, and rate = 1/2 


~ where k is the number of message bits and 7 is the block length. The generator polynomials 


for the two convolutional encoders in the regular/irregular turbo codes are as follows: 


Encoder 1: g(D) = 1 + D* 
Encoder 2: g(D) = 1+ D + D? + D? + D* 


Figure 10.33 also includes the corresponding theoretical limit on channel capacity for code 
rate r = 1/2. 
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FicureE 10.33 Noise performances of regular turbo code, irregular turbo code and irregular 
low-density parity-check (LDPC) code, compared to the Shannon limit for code rate r = 1/2. 
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Based on the results presented in Figure 10.33, we may make the following 
observations: 


> The irregular LDPC code outperforms the regular turbo code in that it comes closer 
to Shannon’s theoretical limit by 0.175 dB. 


» Among the three codes displayed therein, the irregular turbo code is the best in that 
it is just 0.213 dB away from Shannon’s theoretical limit. 


{| 10.12 Summary and Discussion 





In this chapter, we studied error-control coding techniques that have established themselves 
as indispensable tools for reliable digital communication over noisy channels. The effect 
of errors occurring during transmission is reduced by adding redundancy to the data prior 
to transmission in a controlled manner. The redundancy is used to enable a decoder in the 
receiver to detect and correct errors. 

Error-control coding techniques may be divided into two broadly defined families: 


1. Algebraic codes, which rely on abstract algebraic structure built into the design 
of the codes for decoding at the receiver. Algebraic codes include Hamming codes, 
maximal-length codes, BCH codes, and Reed-Solomon codes. These particular 
codes share two properties: 

Linearity property, the sum of any two code words in the code is also a code word. 
Cyclic property, any cyclic shift of a code word is also a code word in the code. 
Reed-Solomon codes are very powerful codes, capable of combatting both random 
and burst errors; they find applications in difficult environments such as deep-space 
communications and compact discs. 

Probabilistic codes, which rely on probabilistic methods for their decoding at the 
receiver. Probabilistic codes include trellis codes, turbo codes, and low-density parity- 
check codes. In particular, the decoding is based on one or the other of two basic 
methods, as summarized here: 

Soft input—hard output, which is exemplified by the Viterbi algorithm that performs 
maximum likelihood sequence estimation in the decoding of trellis codes. 

Soft input-soft output, which is exemplified by the BCJR algorithm that performs 
maximum 4 posteriori estimation on a bit-by-bit basis in the decoding of turbo codes, 
or a special form of the sum-product algorithm in the decoding of low-density parity- 
check codes. 


P 


Trellis codes combine linear convolutional encoding and modulation to permit significant 
coding gains over conventional uncoded multilevel modulation without sacrificing band- 
width efficiency. Turbo codes and low-density parity-check codes share the following 
properties: 


> Random encoding of a linear block kind. 
® Error performance within a hair’s breadth of Shannon’s theoretical limit on channel 
capacity in a physically realizable fashion. 


In practical terms, turbo codes and Jow-density parity-check codes have made it possible 
to achieve coding gains on the order of 10 dB, which is unmatched previously. These 
coding gains may be exploited to dramatically extend the range of digital communication 
receivers, substantially increase the bit rates of digital communication systems, or signifi- 
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cantly decrease the transmitted signal energy per symbol. These benefits have significant 
implications for the design of wireless communications and deep-space communications, 
just to mention two important applications of digital communications. Indeed, turbo codes 
have already been standardized for use on deep-space communication links and wireless 
communication systems. 
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Low-density parity-check (LDPC) codes were originally discovered by Gallager (1962, 
1963). They were rediscovered independently by MacKay and Neal (1995); see also 
MacKay (1999). 
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i PROBLEMS 


Soft-Decision Coding 


10.1 Consider a binary input Q-ary output discrete memoryless channel. The channel is said 
to be symmetric if the channel transition probability p(j|#) satisfies the condition: 


p(j|0) = p(Q - 1 - J 1), j=0,1,...,Q0-1 
Suppose that the channel input symbols 0 and 1 are equally likely. Show that the channel 
output symbols are also equally likely; that is, 


1 
] > j a tpeecs 1 
py) Q j=0,1 Q- 


10.2 Consider the quantized demodulator for binary PSK signals shown in Fig. 10.32. The 
quantizer is a four-level quantizer, normalized as in Fig. P10.2. Evaluate the transition 
probabilities of the binary input-quarternary output discrete memoryless channel so 
characterized. Hence, show that it is a symmetric channel. Assume that the transmitted 
signal energy per bit is E,, and the additive white Gaussian noise has zero mean and 
power spectral density No/2. 


Quantizer 
output 






a  Quantizer 
input 


FIGURE P10.2 


10.3 Consider a binary input AWGN channel, in which the binary symbols 1 and 0 are equally 
likely. The binary symbols are transmitted over the channel by means of phase-shift 
keying. The code symbol energy is E, and the AWGN has zero mean and power spectral 
density No/2. Show that the channel transition probability is given by 


(y|0) = tes) = yy 2B" o < y<% 
piy Vin P| T3 \¥ No/ P y 


Linear Block and Cyclic Codes 


10.4 In a single-parity-check code, a single parity bit is appended to a block of k message bits 
(m, ma, - - - , Ma). The single parity bit b, is chosen so that the code word satisfies the 
even parity rule: 





m +m +--+ +m +b = 0, mod 2 


For k = 3, set up the 2* possible code words in the code defined by this rule. 


10.5 Compare the parity-check matrix of the (7, 4) Hamming code considered in Example 
10.2 with that of a (4, 1) repetition code. 
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10.6 Consider the (7, 4) Hamming code of Example 10.2. The generator matrix G and the 
parity-check matrix H of the code are described in that example. Show that these two 
matrices satisfy the condition 


HG7 = 0 
10.7 (a) For the (7, 4) Hamming code described in Example 10.2, construct the eight code 
words in the dual code. 
(b) Find the minimum distance of the dual code determined in part (a). 


10.8 Consider the (5, 1) repetition code of Example 10.1. Evaluate the syndrome s for the 
following error patterns: 
(a) All five possible single-error patterns 
(b) All 10 possible double-error patterns 

10.9 For an application that requires error detection only, we may use a nonsystematic code. 
In this problem, we explore the generation of such a cyclic code. Let g(X) denote the 
generator polynomial, and m(X) denote the message polynomial. We define the code 
polynomial c(X) simply as 

c(X) = m(X)g(X) 
Hence, for a given generator polynomial, we may readily determine the code words in 
the code. To illustrate this procedure, consider the generator polynomial for a (7, 4) 
Hamming code: 
g(X) =1+X+ xX? 

Determine the 16 code words in the code, and confirm the nonsystematic nature of the 
code. 

10.10 The polynomial 1 + X” has 1 + X + X? and 1 + X? + X? as primitive factors. In 
Example 10.3, we used 1 + X + X? as the generator polynomial for a (7, 4) Hamming 
code. In this problem, we consider the adoption of 1 + X? + X? as the generator 
polynomial. This should lead to a (7, 4) Hamming code that is different from the code 
analyzed in Example 10.3. Develop the encoder and syndrome calculator for the gen- 
erator polynomial: 


g(X) = 1+ X +X? 


Compare your results with those in Example 10.3. 
10.11 Consider the (7, 4) Hamming code defined by the generator polynomial 


g(X)=1+X+ X? 


The code word 0111001 is sent over a noisy channel, producing the received word 
0101001 that has a single error. Determine the syndrome polynomial s(X) for this re- 
ceived word, and show that it is identical to the error polynomial e(X). 


10.12 The generator polynomial of a (15, 11) Hamming code is defined by 
e(X)=14+X+ Xt 
Develop the encoder and syndrome calculator for this code, using a systematic form for 
the code. 


10.13 Consider the (15, 4) maximal-length code that is the dual of the (15, 11) Hamming code 
of Problem 10.12. Do the following: 
(a) Find the feedback connections of the encoder, and compare your results with those 
of Table 7.1 on maximal-length codes presented in Chapter 7. 
(b) Find the generator polynomial g(X); hence, determine the output sequence assuming 
the initial state 0001. Confirm the validity of your result by cycling the initial state 
through the encoder. 
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10.14 Consider the (31, 15) Reed-Solomon code. 
(a) How many bits are there in a symbol of the code? 
(b) What is the block length in bits? 
(c) What is the minimum distance of the code? 
(d) How many symbols in error can the code correct? 


Convolutional Codes 


10.15 A convolutional encoder has a single-shift register with two stages, (i.e., constraint length 
K = 3), three modulo-2 adders, and an output multiplexer. The generator sequences of 
the encoder are as follows: 


g” = (1, 0, 1) 
g” = (1, 1, 0) 
g = (1, 1, 1) 


Draw the block diagram of the encoder. 
Note: For Problems 10.16-10.23, the same message sequence 10111... és used so that 
we may compare the outputs of different encoders for the same input. 

10.16 Consider the rate r = 1/2, constraint length K = 2 convolutional encoder of Fig. P10.16. 
The code is systematic. Find the encoder output produced by the message sequence 
10111.... 


Output 


Input 





Flip-flop 


Figure P10.16 


10.17 Figure P10.17 shows the encoder for a rate r = 1/2, constraint length K = 4 convolu- 
tional code. Determine the encoder output produced by the message sequence 10111.... 





Output 

















FIGURE P10.17 


10.18 


10.19 


10.20 


10.21 


10.22 


10.23 


10.24 


10.25 


10.26 


10.27 


Problems 699 


Consider the encoder of Fig. 10.136 for a rate r = 2/3, constraint length K = 2 con- 
volutional code. Determine the code sequence produced by the message sequence 
10111.... 

Construct the code tree for the convolutional encoder of Fig. P10.16. Trace the path 

through the tree that corresponds to the message sequence 10111... , and compare the 

encoder output with that determined in Problem 10.16. 

Construct the code tree for the encoder of Fig. P10.17. Trace the path through the tree 

that corresponds to the message sequence 10111..... Compare the resulting encoder 

output with that found in Problem 10.17. 

Construct the trellis diagram for the encoder of Fig. P10.17, assuming a message sequence 

of length 5. Trace the path through the trellis corresponding to the message sequence 

10111... . Compare the resulting encoder output with that found in Problem 10.17. 

Construct the state diagram for the encoder of Fig. P10.17. Starting with the all-zero 

state, trace the path that corresponds to the message sequence 10111... , and compare 

the resulting code sequence with that determined in Problem 10.17. 

Consider the encoder of Fig. 10.135. 

(a) Construct the state diagram for this encoder. 

(b) Starting from the all-zero state, trace the path that corresponds to the message se- 
quence 10111. . . . Compare the resulting sequence with that determined in Problem 
10.18. 

By viewing the minimum shift keying (MSK) scheme as a finite-state machine, construct 

the trellis diagram for MSK. (A description of MSK is presented in Chapter 6.) 

The trellis diagram of a rate-1/2, constraint length-3 convolutional code is shown in 

Figure P10.25. The all-zero sequence is transmitted, and the received sequence is 

100010000. . . . Using the Viterbi algorithm, compute the decoded sequence. 









State 
00 00 00 00 00 
N11 N11 N1 
N N N 
01 >` 
10 
\ \ \ \ 
io 10 10 io lo 10 10 
\ \ \ \ 
11 pape ~ ey 2 
01 01 01 
FIGURE P10.25 


Consider a rate-1/2, constraint length-7 convolutional code with free distance diee = 10. 
Calculate the asymptotic coding gain for the following two channels: 


(a) Binary symmetric channel 
(b) Binary input AWGN channel 


In Section 10.6 we described the Viterbi algorithm for maximum likelihood decoding of 
a convolutional code. Another application of the Viterbi algorithm is for maximum 
likelihood demodulation of a received sequence corrupted by intersymbol interference 
due to a dispersive channel. Figure P10.27 shows the trellis diagram for intersymbol 
interference, assuming a binary data sequence. The channel is discrete, described by the 
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10.28 


finite impulse response (1, 0.1). The received sequence is (1.0, —0.3, —0.7, 0,...). Use 
the Viterbi algorithm to determine the maximum likelihood decoded version of this 
sequence. 





Figure P10.27 


Figure P10.28 depicts 32-QAM cross constellation. Partition this constellation into eight 
subsets, At each stage of the partitioning, indicate the within-subset (shortest) Euclidean 
distance, 


oe o o 


FIGURE P10.28 


10.29 As explained in the Introduction to this chapter, channel coding can be used to reduce 


10.30 


the E,/No required for a prescribed error performance or reduce the size of the receiving 
antenna for a prescribed E,/No. In this problem we explore these two practical benefits 
of coding by revisiting Example 8.2 in Chapter 8 on the downlink power calculations 
for a domestic satellite communication system. In particular, we now assume that the 
design of the downlink includes the use of a coding scheme consisting of a rate-1/2 
convolutional encoder with length K = 7 and Viterbi decoding. The coding gain of this 
scheme is 5.1 dB, assuming the use of soft quantization. Hence do the following: 

(a) Recalculate the required E,/Np ratio of the system. 

(b) Assuming that the required E,/No ratio remains unchanged, calculate the reduction 
in the size of the receiving dish antenna that is made possible by the use of this 
coding scheme in the downlink. 

Unlike the convolutional codes considered in this chapter, we recall from Chapter 6 that 

the convolutional code used in the voiceband modem V.32 modem is nonlinear. Figure 

P10.30 shows the circuit diagram of the convolutional encoder used in this modem; it 

uses modulo-2 multiplication and gates in addition to modulo-2 additions and delays. 

Explain the reason for nonlinearity of the encoder in Fig. P10.30, and use an example 

to illustrate your explanation. 
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Flip-flop 











FiGurE P10.30 


Turbo Codes 


10.31 Let r® = p/q, and r® = piq, be the code rates of RSC encoders 1 and 2 in the turbo 
encoder of Fig. 10.25. Find the code rate of the turbo code. 


10.32 The feedback nature of the constituent codes in the turbo encoder of Fig. 10.25 has the 
following implication: A single bit error corresponds to an infinite sequence of channel 
errors. Illustrate this phenomenon by using a message sequence consisting of symbol 1 
followed by an infinite number of symbols 0. 


10.33 Consider the following generator matrices for rate 1/2 turbo codes: 
1+D+D* 

1+ D? 

1+ D?+ D? | 


4-state encoder: g(D) = |a, 


1+ D* 
aes 
(a) Construct the block diagram for each one of these RSC encoders. 
(b) Setup the parity-check equation associated with each encoder. 
10.34 The turbo encoder of Fig. 10.25 involves the use of two RSC encoders. 
(a) Generalize this encoder to encompass a total of M interleavers. 


(b) Construct the block diagram of the turbo decoder that exploits the M sets of parity- - 
check bits generated by such a generalization. 
10.35 Turbo decoding relies on the feedback of extrinsic information. The fundamental prin- 
ciple adhered to in the turbo decoder is to avoid feeding a decoding stage information 


that stems from the stage itself. Explain the justification for this principle in conceptual 
terms. 


8-state encoder: g(D) = E 





16-state encoder: g(D) = E 


10.36 Suppose a communication receiver consists of two components, a demodulator and a 
decoder. The demodulator is based on a Markov model of the combined modulator and 
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channel, and the decoder is based on a Markov model of a forward error correction 
code. Discuss how the turbo principle may be applied to construct a joint demodulator/ 
decoder for this system. 


Computer Experiment 


10.37 In this experiment we continue the investigation into turbo codes presented in Section 
10.9 by evaluating the effect of block size on the noise performance of the decoder, 
As before, the two convolutional encoders of the turbo encoder are as follows; 


Encoder 1: [1, 1, 1] 
Encoder 2: [1, 0, 1] 


The transmitted E,/No is 1 dB. The block errors to termination are prescribed not to 
exceed 15. - 

With this background information, plot the bit error rate of the turbo decoder 
versus the number of iterations for two different block (i.e., interleaver) sizes: 200 and 
400. 


PROBABILITY THEORY | 





| Al.1 Probabilistic Concepts 


Probability theory is rooted in phenomena that, explicitly or implicitly, can be modeled 
by an experiment with an outcome that is subject to chance. Moreover, if the experiment 
is repeated, the outcome can differ because of the influence of an underlying random 
phenomenon or chance mechanism. Such an experiment is referred to as a random exper- 
iment. For example, the experiment may be the observation of the result of tossing a fair 
coin. In this experiment, the possible outcomes of a trial are “heads” or “tails.” 

To be more precise in the description of a random experiment, we ask for three 
features: 


1. The experiment is repeatable under identical conditions. 

2. On any trial of the experiment, the outcome is unpredictable. 

3. For a large number of trials of the experiment, the outcomes exhibit statistical reg- 
ularity; that is, a definite average pattern of outcomes is observed if the experiment 
is repeated a large number of times. 


W RELATIVE-FREQUENCY APPROACH 


Let event A denote one of the possible outcomes of a random experiment. For example, 
in the coin-tossing experiment, event A may represent “heads.” Suppose that in » trials of 
the experiment, event A occurs N,,(A) times. We may then assign the ratio N,,(A)/n to the 
event A. This ratio is called the relative frequency of the event A. Clearly, the relative 
frequency is a nonnegative real number less than or equal to one. That is to say, 





0s <1 (A1.1) 


If event A occurs in none of the trials, N,,(A)/z = 0. If, on the other hand, event A occurs 
in all the z trials, N,(A)/n = 1. 

We say that the experiment exhibits statistical regularity if for any sequence of n 
trials the relative frequency N,,(A)/# converges to the same limit as 2 becomes large. It thus 
seems natural for us to define the probability of event A as 


P(A) = lim (~a) (A1.2) 


now 


The limit shown in Equation (A1.2) should not be viewed in a mathematical sense. Rather, 
we think of Equation (A1.2) as a statement that the probability of an event is the long- 
term proportion of times that a particular event A occurs in a long sequence of trials. For 
example, in the coin-tossing experiment, we may expect.that out of a million tosses of a 
fair coin, about one half of them will show up heads. 

The probability of an event is intended to represent the likelihood that a trial of the 
experiment will result in the occurrence of that event. For many engineering applications 
and games of chance, the use of Equation (A1.2) to define the probability of an event is 
acceptable. However, for many other applications, this definition is inadequate. Consider, 
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for example, the statistical analysis of the stock market: How are we to achieve repeata. 
bility of such an experiment? A more satisfying approach is to state the properties thar 
any measure of probability is expected to have, postulate them as axioms, and then use 
relative-frequency interpretations to justify them. 


= AXIOMS OF PROBABILITY 


When we perform a random experiment, it is natural for us to be aware of the various 
outcomes that are likely to arise. In this context, it is convenient to think of an experiment 
and its possible outcomes as defining a space and its points. With the kth outcome of the 
experiment, say, we associate a point called the sample point, which we denote by s,. The 
totality of sample points corresponding to the aggregate of all possible outcomes of the 
experiment is called the sample space, which we denote by S. An event corresponds to 
either a single sample point or a set of sample points. In particular, the entire sample space 
S is called the sure event; the null set Ø is called the null or impossible event; and a single 
sample point is called an elementary event. 

Consider, for example, an experiment that involves the throw of a die, In this ex- 
periment there are six possible outcomes: the showing of one, two, three, four, five, and 
six dots on the upper face of the die. By assigning a sample point to each of these possible 
outcomes, we have a one-dimensional sample space that consists of six sample points, as 
shown in Figure A1.1. The elementary event describing the statement “a six shows” cor- 
responds to the sample point {6}. On the other hand, the event describing the statement 
“an even number of dots shows” corresponds to the subset {2, 4, 6} of the sample space. 
Note that the term event is used interchangeably to describe the subset or the statement. 

We are now ready to make a formal definition of probability. A probability system 
consists of the triple: 


1. A sample space S of elementary events (outcomes). 

2. A class 8 of events that are subsets of S. 

3. A probability measure P(-) assigned to each event A in the class 8, which has the 
following properties: 


(i) 


P(S) = 1 (A1.3) 
(ii) 
0<P(A)<1 (A1.4) 
(iii) If A + B is the union of two mutually exclusive events in the class €, then 
P(A + B) = P(A) + P(B) (A1.5) 


Properties (i), (ii), and (iii) are known as the axioms of probability. Axiom (i) states that 
the probability of the sure event is unity. Axiom (ii) states that the probability of an event 
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FIGURE Al.1 Sample space for the experiment of throwing a die. 
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is a nonnegative real number that is less than or equal to unity. Axiom (iii) states that the 
probability of the union of two mutually exclusive events is the sum of the probabilities 
of the individual events. These three axioms are sufficient to deal with experiments with 
finite sample spaces. 

Although the axiomatic approach to probability theory is abstract in nature, all three 
axioms have relative-frequency interpretations of their own. Axiom (ii) corresponds to 
Equation (A1.1). Axiom (i) corresponds to the limiting case of Equation (A1.1) when the 
event A occurs in all the » trials. To interpret axiom (ili), we note that if event A occurs 
N,(A) times in v trials and event B occurs N,(B) times, then the union event “A or B” 
occurs in N,(A) + N,(B) trials (since A and B can never occur on the same trial). Hence, 
N,(A + B) = N,(A) + N,(B), and so we have 

NAA + B) _ N,(A) _ N,(B) 


4 
n n n 








which has a mathematical form similar to that of axiom (iii). 
Axioms (i), (ii), and (iii) constitute an implicit definition of probability. We may use 
these axioms to develop some other basic properties of probability, as described next. 


Property 1 
P(A) = 1 — P(A) (A1.6) 


where A (denoting “not A”) is the complement of event A. 


The use of this property helps us investigate the nonoccurrence of an event. To prove 
it, we express the sample space S as the union of two mutually exclusive events A and A: 
S=A+A 
Then, the use of axioms (i) and (iii) yields 
= P(A) + P(A) 


from which Equation (A1.6) follows directly. 


Property 2 
If M mutually exclusive events Ay, An, . . . , Ay bave the exhaustive property 
Ai + Aye sn Ay = S (A1.7) 
then 
P(A,) + P(A) + +++ + P(Ay) = 1 (A1.8) 


To prove this property, we first use axiom (i) in Equation (A1.7), and so write 
P(A, + A, +--+ + Am) =1 
Next, we generalize axiom (iii) by writing 


P(A; + A, + +++ + Am) = P(A,) + P(A2) +--+ + P(A) 
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Hence, the result of Equation (A1.8) follows. When the M events are equally likely (i.e, 
they have equal probabilities of occurrence), then Equation (A1.8) simplifies as 


P(A;) = i=1,2,...,M 


M? 


Property 3 


When events A and B are not mutually exclusive, then the probability of the union event 
“A or B” equals 


P(A + B) = P(A) + P(B) — P(AB) (A1.9} 
where P(AB) is the probability of the joint event “A and B.” 


The probability P(AB) is called a joint probability. It has the following relative- 
frequency interpretation: ` 


æ n 


P(AB) = im (240 (A1.10) 


where N,,(AB) denotes the number of times the events A and B occur simultaneously in n 
trials of the experiment. Axiom (iii) is a special case of Equation (A1.9); when A and B 
are mutually exclusive, P(AB) is‘zero, and Equation (A1.9) reduces to the same form as 
Equation (A1.5). 


© CONDITIONAL PROBABILITY 


Suppose we perform an experiment that involves a pair of events A and B. Let P(B |A) 
denote the probability of event B, given that event A has occurred. The probability 
P(B|A) is called the conditional probability of B given A. Assuming that A has nonzero 
probability, the conditional probability P(B |A) is defined by 
P(AB) 
P(B|A) PIA) (A1.11) 
where P({AB) is the joint probability of A and B. 

We justify the definition of conditional probability given in Equation (A1.11) by 
presenting a relative-frequency interpretation of it. Suppose that we perform an experiment 
and examine the occurrence of a pair of events A and B. Let N,,(AB) denote the number 
of times the joint event AB occurs in 7 trials. Suppose that in the same 7 trials, the event 
A occurs N,„(A) times. Since the joint event AB corresponds to both A and B occurring, it 
follows that N,,(A) must include N,,(AB). In other words, we have 


N,(AB) _ 


May 1 


The ratio N,,(AB)/N,(A) represents the relative frequency of B given that A has occurred. 
For large x, this ratio equals the conditional probability P(B |A); that is, 
N,{AB) 


KEMIE tin ( N.A) ) 
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or, equivalently, 


P(B|A) = lim 


ft 


N,(AB)/n 
N, (Ayn 


Recognizing that 


P(AB) = lim (24m) 


n 


nwo 


and 


P(A) = lim (=a) 


n 


noo 


the result of Equation (A1.11) follows. 
We may rewrite Equation (A1.11) as 


P(AB) = P(B|A)P(A) (A1.12) 
It is apparent that we may also write 
P(AB) = P(A|B)P(B) (A1.13) 


Accordingly, we may state that the joint probability of two events may be expressed as 
the product of the conditional probability of one event given the other, and the elementary 
probability of the other. Note that the conditional probabilities P(B| A) and P(A |B) have 
essentially the same properties as the various probabilities previously defined. 

Situations may exist where the conditional probability P(A | B) and the probabilities 
P(A) and P(B) are easily determined directly, but the conditional probability P(B|A) is 
desired. From Equations (A1.12) and (A1.13), it follows that, provided P(A) # 0, we may 
determine P(B|A) by using the relation 

P(A|B)P(B) 
P(B|A) PIA) (A1.14) 

This relation is a special form of Bayes’ rule. 

Suppose that the conditional probability P(B| A) is simply equal to the elementary 
probability of occurrence of event B, that is, 


P(B|A) = P(B) 


Under this condition, the probability of occurrence of the joint event AB is equal to the 
product of the elementary probabilities of the events A and B: 


P(AB) = P(A)P(B) 
so that 
P(A|B) = P(A) 


That is, the conditional probability of event A, assuming the occurrence of event B, is 
simply equal to the elementary probability of event A. We thus see that in this case a 
knowledge of the occurrence of one event tells us no more about the probability of oc- 
currence of the other event than we knew without that knowledge. Events A and B that 
satisfy this condition are said to be statistically independent. 
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i Al.2 Random Variables 


It is customary, particularly when using the language of sample space, to think of the 
outcome of an experiment as a variable that can wander over the set of sample points and 
whose value is determined by the experiment. A function whose domain is a sample space 
and whose range is some set of real numbers is called a random variable of the experiment, 
However, the term random variable is somewhat confusing. First, the word random is not 
used in the sense of equal probability of occurrence, for which it should be reserved, 
Second, the word variable does not imply dependence (on the experimental outcome), 
which is an essential part of the meaning. Nevertheless, the term is so deeply imbedded in 
the literature of probability that its usage has persisted. 

When the outcome of an experiment is s, the random variable is denoted as X(s) or 
simply X. For example, the sample space representing the outcomes of the throw of a die 
is a set of six sample points that may be taken to be the integers 1, 2,..., 6. Then if we 
identify the sample point k with the event that k dots show when the die is thrown, the 
function X(k) = k is a random variable such that X(k) equals the number of dots that 
show when the die is thrown. In this example, the random variable takes only a discrete 
set of values. In such a case, we say that we are dealing with a discrete random variable, 
More precisely, the random variable X can take only a finite number of values in any finite 
observation interval. If, however, the random variable X can take any value in a whole 
observation interval, X is called a continuous random variable. For example, the random 
variable that represents the amplitude of a noise voltage at a particular instant of time is 
a continuous random variable because it may take any value between plus and minus 
infinity. 

To proceed further, we need a probabilistic description of random variables that 
works equally well for discrete as well as continuous random variables. Let us consider 
the random variable X and the probability of the event X = x. We denote this probability 
by P(X < x). It is apparent that this probability is a function of the dummy variable x. To 
simplify the notation, we write 


Fy(x) = P(X = x) (A1.15) 


The function F(x) is called the cumulative distribution function (cdf) or simply the dis- 
tribution function of the random variable X. Note that Fx(x) is a function of x, not of the 
random variable X. However, it depends on the assignment of the random variable X, 
which accounts for the use of X as subscript. For any point x, the distribution function 
Fy(x) expresses a probability. 

The distribution function Fx(x) has the following properties, which follow directly 
from Equation (A1.15): 


1. The distribution function Fx(x) is bounded between zero and one. 
2. The distribution function Fx(x) is a nondecreasing function of x; that is, 


Fy (x1) = Fx(x2) if x, < x2 (A1.16) 


An alternative description of the probability of the random variable X is often useful. 
This is the derivative of the distribution function, as shown by 


fx(x) = ma F(x) (A1.17) 
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which is called the probability density function (pdf) of the random variable X. Note that 
the differentiation in Equation (A1.17) is with respect to the dummy variable x. The name, 
density function, arises from the fact that the probability of the event x, < X <= x2 equals 


P(xy < X = x) = P(X = x2) — P(X S x) 
= Ex(x2) — Fx(x1) (A1.18) 


= [fete dx 


The probability of an interval is therefore the area under the probability density function 
in that interval. Putting x; = —œ in Equation (A1.18), and changing the notation some- 
what, we readily see that the distribution function is defined in terms of the probability 
density function as follows: 


Fx(x) = B flé) dé (A1.19) 


Since F(%) = 1, corresponding to the probability of a certain event, and Fx({—%) = 0, 
corresponding to the probability of an impossible event, we readily find from Equation 
(A1.18) that 


i Fx(x) dx = 1 (A1.20) 


Earlier we mentioned that a distribution function must always be nondecreasing. This 
means that its derivative or the probability density function must always be nonnegative. 
Accordingly, we may state that a probability density function must always be a nonneg- 
ative function, and with a total area of one. 

Thus far we have focused attention on situations involving a single random variable. 
However, we find frequently that the outcome of an experiment requires several random 
variables for its description. We now consider situations involving two random variables. 
The probabilistic description developed in this way may be readily extended to any number 
of random variables. 

Consider two random variables X and Y. We define the joint distribution function 
Fx x(x, y) as the probability that the random variable X is less than or equal to a specified 
value x and that the random variable Y is less than or equal to a specified value y. The 
variables X and Y may be two separate one-dimensional random variables or the com- 
ponents of a single two-dimensional random variable. In either case, the joint sample space 
is the xy-plane. The joint distribution function Fy y(x, y) is the probability that the outcome 
of an experiment will result in a sample point lying inside the quadrant (—% < X = x, 
~% < Y= y) of the joint sample space. That is, 


Fy yx, y) = P(X = x, Y= y) (A1.21) 


Suppose that the joint distribution function Fy y(x, y) is continuous everywhere, and 
that the partial derivative 


Tatay) (A1.22) 


exists and is continuous everywhere. We call the function fx,y(x, y) the joint probability 
density function of the random variables X and Y. The joint distribution function 


fxxlx, y) = 
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Fy.y(x, y) is a nondecreasing function of both x and y. Therefore, from Equation (A1,22) 
it follows that the joint probability density function fy,y(x, y) is always nonnegative. Also 
the total volume under the graph of a joint probability density function must be unity, as 
shown by 


a ee fxxlé n) dédn = 1 (A1.23) 


The probability density function for a single random variable (X, say) can be ob- 
tained from its joint probability density function with a second random variable (Y, say) 
in the following way. We first note that 


Fle) = | f fere 0) dé dn (A1.24) 


Therefore, differentiating both sides of Equation (A1.24) with respect to x, we get the 
desired relation: 


fete) = | Sexis m dn (A1.25) 


Thus the probability density function fx(x) is obtained from the joint probability density 
function fx y(x, y) by simply integrating it over all possible values of the undesired random 
variable Y. The use of similar arguments in the other dimension yields fy(y). The proba- 
bility density functions f(x) and fy(y) are called marginal densities. Hence, the joint 
probability density function fx,y(x, y) contains all the possible information about the joint 
random variables X and Y. 

Suppose that X and Y are two continuous random variables with joint probability 
density function fx.y(x, y). The conditional probability density function of Y given that 
X = x is defined by 


fily|x) = ae? (A1.26) 


provided that fx(x) > 0, where fx(x) is the marginal density of X. The function Fry |x) 
may be thought of as a function of the variable y, with the variable x arbitrary, but fixed. 
Accordingly, it satisfies all the requirements of an ordinary probability density function, 
as shown by 


foly|x) = 0 


and 


T felylx) dy=1 


If the random variables X and Y are statistically independent, then knowledge of the 
outcome of X can in no way affect the distribution of Y. The result is that the conditional 
probability density function fy(y |x) reduces to the marginal density fy(y), as shown by 


fely|x) = foly) 


In such a case, we may express the joint probability density function of the random vari- 
ables X and Y as the product of their respective marginal densities, as shown by 


fuy(x, y) = fala) frly) 
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In words, we may state that if the joint probability density function of the random variables 


X and Y equals the product of their marginal densities, then X and Y are statistically 
independent. 


Statistical Averages 





Having discussed probability and some of its ramifications, we now seek ways for deter- 
mining the average behavior of the outcomes arising in random experiments. 
The expected value or mean of a random variable X is defined by 


ux = E[X] = iz xfx(x) dx (A1.27) 


where E denotes the statistical expectation operator. That is, the mean uy locates the center 
of gravity of the area under the probability density curve of the random variable X. To 
interpret the expected value jx, we write the integral in the defining Equation (A1.27) as 
the limit of an approximating sum formulated as follows. Let [x,|k = 0, +1, +2,...} 
denote a set of uniformly spaced points on the real line: 


“* (: + i) A, k=0,+1,42,... (A1.28) 
where A is the spacing between adjacent points. We may then rewrite Equation (A1.27) 
as the limiting form of a sum: 


CJ (k+1) A 


E[X] = lim > Xefx(x) dx 
AO kare vk A 


(A1.29 
= lim Š a(n- $< xni) ) 

A+0 k£ 2 2 
For a physical interpretation of the sum on the right-hand side of Equation (A1.29), sup- 
pose that we make independent observations of the random variable X. Let N„(k) denote 
the number of times that the random variable X falls inside the kth bin: 


A 
mo $< Xsm43, k = 0, #1, #2,... 


Then, as the number of observations, », is made large, the ratio N,,(k)/n approaches the 
probability P(x, — A/2 < X = x, + A/2). Accordingly, we may approximate the expécted 
value of the random variable X as 


n 


=—o 


=. E xN,(R), n large 


A k==m 


(A1.30) 


We now recognize the quantity on the right-hand side of Equation (A1.30) simply as the 
sample average. The sum is taken over all the values x,, each of which is weighted by the 
number of times it occurs; the sum is then divided by the total number of observations to 
give the sample average. Indeed, Equation (A1.30) provides the basis for computing the 
expected value E[X]. 
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We next consider a more general situation. Let X denote a random variable, and let 
g(X) denote a function of X defined on the real line. The quantity obtained by letting the 
argument of the function g(X) be a random variable is also a random variable, which we 
denote as 


Y = g(X) (A1.31) 


To find the expected value of the random variable Y, we could of course find the probability 
density function fy(y) and then apply the standard formula 


E[Y] = i. yfvly) dy 


A simpler procedure, however, is to write 


Elg(X)] = is g(x) fix(x) dx (A1,32) 


Indeed, Equation (A1.32) may be viewed as generalizing the concept of expected value to 
an arbitrary function g(X) of a random variable X. 


E MOMENTS 
For the special case of g(X) = X”, using Equation (A1.32) we obtain the nth moment of 
the probability distribution of the random variable X; that is, 


oO 


E[x = T x” fglx) dx (A1.33) 


By far the most important moments of X are the first two moments. Thus putting n = 1 
in Equation (A1.33) gives the mean of the random variable as shown in Eq. (A1.27), 
whereas putting n = 2 gives the mean-square value of X: 


E[X*] = [e x? fxlx) dx (A1.34) 


We may also define central moments, which are simply the moments of the difference 
between a random variable X and its mean px. Thus, the nth central moment is 


E[(X - yxy] = i (x — xY fx(x) dx (A1.35) 


For n = 1, the central moment is, of course, zero, whereas for n = 2 the second central 
moment is referred to as the variance of the random variable X, which is written as 


var[X] = E[(X ~ px] = i (x — px) fle) dx (A1.36) 


The variance of a random variable X is commonly denoted as o%. The square root of the 
variance, namely, ox, is called the standard deviation of the random variable X. 

The variance o% of a random variable X in some sense is a measure of the variable’s 
“randomness.” By specifying the variance ox, we essentially constrain the effective width 
of the probability density function fx(x) of the random variable X about the mean Hx- 
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A precise statement of this constraint is due to Chebyshev. The Chebyshev inequality states 
that for any positive number e€, we have 


o 
P(X ~ ax| = 6) =H (A1.37) 


From this inequality we see that the mean and variance of a random variable give a partial 
description of its probability distribution, hence their common use in practice. 

We note from Equations (A1.34) and (A1.36) that the variance o% and mean-square 
value E[X*] are related by 


ox = E[X? — 2uxX + p3] 
= E[X?] — 2uxE[X] + wx (A1.38) 
= E[X?] — ak 
where, in the second line, we have used the linearity property of the statistical expectation 


operator E. Equation (A1.38) shows that if the mean px is zero, then the variance o% and 
the mean-square value E[X?] of the random variable X are equal. 


E CHARACTERISTIC FUNCTION 


Another important statistical average is the characteristic function 6 (v) of the probability 
distribution of the random variable X, which is defined as the expectation of the complex 
exponential function exp(jvX), as shown by 


ox(v) = Efexp(jvX)] 


a (A1.39) 

= i fx(x) expl jvx) dx 
where v is real and j = V —1. In other words, the characteristic function ¢x(v) is (except 
for a sign change in the exponent) the Fourier transform of the probability density function 
fx(x); the Fourier transform is reviewed in Appendix 2. In this relation we have used 
exp(jvx) rather than exp(—jvx), so as to conform with the convention adopted in proba- 
bility theory. Recognizing that v and x play analogous roles to the variables 27f and t of 
Fourier transforms, respectively, we deduce the following inverse relation from analogy 
with the inverse Fourier transform: 


fie) =z | dv) exp jox) dv (A1.40) 


This relation may be used to evaluate the probability density function fx(x) of the random 
variable X from its characteristic function ¢x(v). 


Joint MoMENTS 


Consider next a pair of random variables X and Y. A set of statistical averages of impor- 
tance in this case is the joint moments, namely, the expected value of X’Y*, where i and 
k may assume any positive integer values. We may thus write 


ev) | [7 wfe, y) dx dy (A1.41) 
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A joint moment of particular importance is the correlation deifned by E[XY], which cor- 
responds to 7 = k = 1 in Equation (A1.41). 
The correlation of the centered random variables X — E[X] and Y — E[Y], that is, 
the joint moment 
cov[XY] = E[(X — E[X])(Y ~ E[Y])] (A1.42) 
is called the covariance of X and Y. Letting ux = E[X] and wy = E[Y], we may expand 
Equation (A1.42) to obtain the result 
cov[XY] = E[XY] — puxpy (A1.43) 
Let o% and a4 denote the variances of X and Y, respectively. Then the covariance of X 
and Y, normalized with respect to oxoy, is called the correlation coefficient of X and Y; 
= cov[XY] 
oxy 


(A1.44) 


We say that the two random variables X and Y are uncorrelated if and only if their 
covariance is zero, that is, if and only if 


cov[XY] = 0 
We say that they are orthogonal if and only if their correlation is zero, that is, if and 
only if 

E[XY] = 0 
From Equation (A1.43) we observe that if one of the random variables X and Y or both 
have zero means, and if they are orthogonal, then they are uncorrelated, and vice versa. 


Note also that if X and Y are statistically independent, then they are uncorrelated; however, 
the converse of this statement is not necessarily true. 





REPRESENTATION OF 
| SIGNALS AND SYSTEMS 


| A2.1 Fourier Analysis 


Let g(t) denote a nonperiodic deterministic signal, expressed as some function of time t. 
By definition, the Fourier transform of the signal g(t) is given by the integral 


Gif) = | ete) exp(-i2aft) de (A2.1) 


where j = V—1, and the variable f denotes frequency. Given the Fourier transform G(f), 
the original signal g(t) is recovered exactly using the formula for the inverse Fourier 
transform: 


att) = [GIP expli2afe) df (42.2) 


Note that in Equations (A2.1) and (A2.2) we have used a lowercase letter to denote the 
time function and an uppercase letter to denote the corresponding frequency function. The 
functions g(f) and G(f) are said to constitute a Fourier-transform pair. 

For the Fourier transform of a signal g(t) to exist, it is sufficient but not necessary 
that g(t) satisfies three conditions known collectively as Dirichlet’s conditions: 


1. The function g(t) is single-valued, with a finite number of maxima and minima in 
any finite time interval, ; 

2. The function g(t) has a finite number of discontinuities in any finite time interval. 

3. The function g(t) is absolutely integrable, that is, 


[lan ar<e 


We may safely ignore the question of the existence of the Fourier transform of a time 
function g(t) when it is an accurately specified description of a physically realizable signal. 
In other words, physical realizability is a sufficient condition for the existence of a Fourier 
transform. Indeed, we may go one step further and state that all energy signals, that is, 
signals g(t) for which 


[lew a <x 
are Fourier transformable. 

The Fourier transform provides the mathematical tool for measuring the frequency 
content, or spectrum, of a signal. For this reason, the terms Fourier transform and spectrum 
are often used interchangeably. Thus, given a signal g(t) with Fourier transform G(f), we 
may refer to G(f) as the spectrum of the signal g(t). By the same token, we refer to | G(f)| 
as the magnitude spectrum of the signal g(t), and refer to arg {G(f)} as its phase spectrum. 
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PROPERTIES OF THE FOURIER TRANSFORM 


It is useful to have insight into the relationship between a time function g(t) and its Fourier 
transform G(f), and also into the effects that various operations on the function g(t) have 
on the transform G(f). This may be achieved by examining certain properties of the Fourier 
transform, which are summarized in Table A6.2. 


= Drac DELTA FUNCTION 


Strictly speaking, the theory of the Fourier transform is applicable only to time functions 
that satisfy the Dirichlet conditions. Such functions include energy signals. However, it 
would be highly desirable to extend this theory in two ways: 


1. To combine the Fourier series and Fourier transform into a unified theory, so that 
the Fourier series may be treated as a special case of the Fourier transform. 

2. To include power signals (i.e., signals for which the average power is finite) in the 
list of signals to which we may apply the Fourier transform. 


It turns out that both of these objectives can be met through the “proper use” of the Dirac 
delta function, or unit impulse. 

The Dirac delta‘function or just delta function, denoted by 6(f), is defined as having 
zero amplitude everywhere except at t = 0, where it is infinitely large in such a way that 
it contains unit area under its curve; that is, 


6(t) = 0, t#0 (A2.3) 


and 
f a(t) dt = 1 (A2.4) 


An implication of this pair of relations is that the delta function 6(t) must be an even 
function of time t, which is centered at t = 0. 

For the delta function to have meaning, however, it has to appear as a factor in the 
integrand of an integral with respect to time and then, strictly speaking, only when the 
other factor in the integrand is a continuous function of time. Let g(t) be such a function, 
and consider the product of g(t) and the time-shifted delta function 6(t — to). In light of 
the two defining equations, Equations (A2.3) and (A2.4), we may express the integral of 
this product as follows: 


__ Blt) Bt — to) dt = g(to) (A2.5) 
The operation indicated on the left-hand side of this equation sifts out the value g(to) of 
the function g(t) at time t = tf, where ~% < t < œ. Accordingly, Equation (A2.5) is 
referred to as the sifting property of the delta function. This property is sometimes used 
as the defining equation of a delta function; in effect, it incorporates Equations (A2.3) and 
(A2.4) into a single relation. 
Noting that the delta function 6(t) is an even function of t, we may rewrite Equation 
(A2.5) so as to emphasize its resemblance to the convolution integral, as shown by 


a 


g(t) ôt — 7) dr = git) (A2.6) 
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In words, the convolution of any function with the delta function leaves that function 
unchanged. We refer to this statement as the replication property of the delta function. 
It is important to realize that no function in the ordinary sense has the two properties 
of Equations (A2.3) and (A2.4) or the equivalent sifting property of Equation (A2.5). 
However, we can imagine a sequence of functions that have progressively taller and thinner 
peaks at t = 0, with the area under the curve remaining equal to unity, whereas the value 
of the function tends to zero at every point except t = 0, where it tends to infinity. That 
is, we may view the delta function as the limiting form of a pulse of unit area as the 
duration of the pulse approaches zero. It is immaterial what sort of pulse shape is used. 


m FOURIER TRANSFORMS OF PERIODIC SIGNALS 


It is well known that by using the Fourier series, a periodic signal can be represented as a 
sum of complex exponentials. Also, in a limiting sense, Fourier transforms can be defined 
by complex exponentials. Therefore, it seems reasonable to represent a periodic signal in 
terms of a Fourier transform, provided that this transform is permitted to include delta 
functions, 

Consider then a periodic signal gr (t) of period Ty. We can represent gr (t) in terms 
of the complex exponential Fourier series: 


Enlt) = È cn expl j2anfot) (A2.7) 

where c, is the complex Fourier coefficient defined by 
1 (te? 
T To J-t9r2 
‘and fo is the fundamental frequency defined as the reciprocal of the period To; that is, 


1 
fo= T, (A2.9) 


&r,(t) exp(—j2anfo) dt (A2.8) 


Cn 


Let g(t) be a pulselike function, which equals gr,(ż) over one period and is zero elsewhere; 
that is 


To 0 
g(e) = 42m -75t (A2.10) 
0 elsewhere 


The periodic signal g7,(t) may now be expressed in terms of the function g(t) as an infinite 
summation, as shown by 


Enlt) = > gt- mTo) (A2.11) 


m=% 


Based on this representation, we may view g(t) as a generating function, which generates 
the periodic signal g,(t). 

The function g(t) is Fourier transformable. Accordingly, we may rewrite the formula 
for the complex Fourier coefficient as follows: 


cn = fo | _ ett) exp(—i2mnfot) dt 
= foG(fo) (A2.12) 
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where Ginfo) is the Fourier transform of g(t) evaluated at the frequency #9. We may thus 
rewrite the formula for the reconstruction of the periodic signal £1,(t) as 


æ% 


Enlt) = fo È Glnfo) explj2mnfot) (A2.13) 


n=% 


or, equivalently, in light of Equation (A2.11) 


D gt- mT) = fo © Glnfo) exp(j2mnfot) (A2.14) 
Equation (A2.14) is one form of Poisson’s sum formula. 

It is of interest to observe that the function g(t), which constitutes one period of the 
periodic signal g7,(t), has a continuous spectrum defined by G(f). On the other hand, the 
period signal g7,(¢) itself has a discrete spectrum. We conclude, therefore, that periodicity 
in the time domain has the effect of changing the frequency-domain description or spec- 
trum of the signal into a discrete form defined at integer multiples of the fundamental 


frequency. 


a FOURIJER-TRANSFORM PAIRS 


Table A6.3 presents a listing of some commonly used Fourier-transform pairs, the deri- 
vations of which follow from the material just presented. 


TRANSMISSION OF SIGNALS THROUGH LINEAR SYSTEMS 


A system refers to any physical device that produces an output signal in response to an 
input signal. It is customary to refer to the input signal as the excitation and to the output 
signal as the response. In a linear system, the principle of superposition holds; that is, the 
response of a linear system to a number of excitations applied simultaneously is equal to 
the sum of the responses of the system when each excitation is applied individually. 

In the time domain, a linear system is described in terms of its impulse response, 
which is defined as the response of the system (with zero initial conditions) to a unit impulse 
or delta function 8(t) applied to the input of the system. If the system is time invariant, 
then the shape of the impulse response is the same no matter when the unit impulse is 
applied to the system. Thus, assuming that the unit impulse or delta function is applied at 
time t = 0, we may denote the impulse response of a linear time-invariant system by b(t). 
Let this system be subjected to an arbitrary excitation x(t). The response, y(t), of the system 
is defined in terms of the impulse response h(t) by 


y(t) = r x(rh{t — 7) dr (A2.15) 


=A 


which is called the convolution integral. Equivalently, we may write 
y(t) = [ h(r) x(t — 1) dr (A2.16) 


Hence, convolution is commutative. 

In the convolution integral, three different time scales are involved: excitation time 
T, response time t, and system-memory time t — T. This relation is the basis of time-domain 
analysis of linear time-invariant systems. According to Equation (A2.15), the present value 
of the response of a linear time-invariant system is a weighted integral over the past history 
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of the input signal, weighted according to the impulse response of the system. Thus the 
impulse response acts as a memory function for the system. 


u FREQUENCY RESPONSE OF LINEAR TIME-INVARIANT SYSTEMS 


Consider a linear time-invariant system of impulse response /(t) driven by a complex 
exponential input of unit amplitude and frequency f, that is, 


x(t) = exp(j2aft) 


Using this excitation in Equation (A2.16), the response of the system is obtained as 


ia h(t) exp[j2af(t — 7)] dr 
expl j27ft) i h(t) exp(—j2afr) dr 


y(t) 
(A2.17) 


Define the frequency response of the system as the Fourier transform of its impulse re- 
sponse, as shown by 


Hf) = ie A(t) exp(—j2aft) dt (A2.18) 


The integral in the last line of Equation (A2.17) is the same as that of Equation (A2.18), 
except that 7 is used in place of t. Hence, we may rewrite Equation (A2,17) in the form 


y(t) = Hif) exp(j2aft) (A2.19) 


The response of a linear time-invariant system to a complex exponential function of fre- 
quency f is, therefore, the same complex exponential function multiplied by a constant 
coefficient H(f). 

The frequency response H(f) is, in general, a complex quantity, so we may express 
it in the form 


Hif) = |H(f)| expli8(f)] (A2.20) 


where | H(f)| is called the magnitude response, and B(f) is the phase, or phase response. 
In the special case of a linear system with a real-valued impulse response h(¢), the frequency 
response H(f) exhibits conjugate symmetry, which means that 


|H(f)| = |H(-f)| 
and 


B) = —B(-f) 


That is, the magnitude response | H( f.) | of a linear system with real-valued impulse response 
is an even function of frequency, whereas the phase 8(f) is an odd function of frequency. 

In some applications, it is preferable to work with the logarithm of H(f) expressed 
in polar form rather than with H(f) itself. Define the natural logarithm 


log H(f) = af) + jB(f) (A2.21) 
where : 


a(f) = log|H(f)| (A2.22) 
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The function a(f) is called the gain of the system. It is measured in nepers, whereas (f) 
is measured in radians. Equation (A2.21) indicates that the gain «(f) and phase B(f) are 
the real and imaginary parts of the (natura!) logarithm of the frequency response H(f), 
respectively. The gain may also be expressed in decibels (dB) by using the definition 


a'(f) = 20 logio| H(f)| (A2.23) 
The two gain functions a(f) and a'(f) are related by 
a'(f) = 8.69a(f) (A2.24) 


That is, 1 neper is equal to 8.69 dB. 


i A2.2 Bandwidth 


The time-domain and frequency-domain descriptions of a signal are inversely related. In 
particular, we may make the following important statements: 


1. If the time-domain description of a signal is changed, the frequency-domain descrip- 
tion of the signal is changed in an inverse manner, and vice versa. This inverse re- 
lationship prevents arbitrary specifications of a signal in both domains. In other 
words, we may specify an arbitrary function of time or an arbitrary spectrum, but 
we cannot specify both of them together. 

If a signal is strictly limited in frequency, the time-domain description of the signal 
will trail on indefinitely, even though its amplitude may assume a progressively 
smaller value. We say a signal is strictly limited in frequency or strictly band limited 
if its Fourier transform is exactly zero outside a finite band of frequencies. The sinc 
pulse 


a 


sin(7t) 


sinc(t) = 


is an example of a strictly band-limited signal. It is also asymptotically limited in 
time, which confirms the opening statement we made for a strictly band-limited 
signal. In an inverse manner, if a signal is strictly limited in time (i.e., the signal is 
exactly zero outside a finite time interval), then the spectrum of the signal is infinite 
in extent, even though the amplitude spectrum may assume a progressively smaller 
value. This behavior is exemplified by a rectangular pulse. Accordingly, we may state 
that a signal cannot be strictly limited in both time and frequency. 


The bandwidth of a signal provides a measure of the extent of significant spectral 
content of the signal for positive frequencies. When the signal is strictly band limited, the 
bandwidth is well defined. For example, the sinc pulse sinc(2W¢) has a bandwidth equal 
to W. However, when the signal is not strictly band limited, as is generally the case, we 
encounter difficulty in defining the bandwidth of the signal. The difficulty arises because 
the meaning of “significant” attached to the spectral content of the signal is mathematically 
imprecise. Consequently, there is no universally accepted definition of bandwidth. Nev- 
ertheless, there are some commonly used definitions for bandwidth, as discussed next. 

When the spectrum of a signal is symmetric with a main lobe bounded by well-defined 
nulls (i.e., frequencies at which the spectrum is zero), we may use the main lobe as the 
basis for defining the bandwidth of the signal. Specifically, if the signal is low-pass (i.e. 
its spectral content is centered around the origin), the bandwidth is defined as one half the 
total width of the main spectral lobe since only one half of this lobe lies inside the positive 
frequency region. For example, a rectangular pulse of duration T seconds has a main 
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spectral lobe of total width 2/T hertz centered at the origin. Accordingly, we may define 
the bandwidth of this rectangular pulse as 1/T hertz. If, on the other hand, the signal is 
band-pass with main spectral lobes centered around +f., where f, is large enough, the 
bandwidth is defined as the width of the main lobe for positive frequencies. This definition 
of bandwidth is called the null-to-null bandwidth. For example, an RF pulse of duration 
T seconds and frequency f. has main spectral lobes of width 2/T hertz centered around 
+ f., where it is assumed that f, is large compared to 1/T. Hence, we may define the null- 
to-null bandwidth of this RF pulse as 2/T hertz. On the basis of the definitions presented 
here, we may state that shifting the spectral content of a low-pass signal by a sufficiently 
large frequency has the effect of doubling the bandwidth of the signal; such a frequency 
translation is attained by using modulation. 

Another popular definition of bandwidth is the 3-dB bandwidth. Specifically, if the 
signal is low-pass, the 3-dB bandwidth is defined as the separation between zero frequency, 
where the magnitude spectrum attains its peak value, and the positive frequency, at which 
the amplitude spectrum drops to 1/V2 of its peak value. For example, the decaying ex- 
ponential exp(—at) has a 3-dB bandwidth of a/27 hertz. If, on the other hand, the signal 
is band-pass, centered at +f., the 3-dB bandwidth is defined as the separation (along the 
positive frequency axis) between the two frequencies at which the magnitude spectrum of 
the signal drops to 1/V2 of the peak value of f.. The 3-dB bandwidth has the advantage 
in that it can be read directly from a plot of the magnitude spectrum. However, it has the 
disadvantage in that it may be misleading if the magnitude spectrum has slowly decreasing 
tails. 

Yet another measure for the bandwidth of a signal is the root mean square (rms) 
bandwidth, which is defined as the square root of the second moment of a properly nor- 
malized form of the squared magnitude spectrum of the signal about a suitably chosen 
point. We assume that the signal is low-pass, so that the second moment may be taken 
about the origin. As for the normalized form of the squared magnitude spectrum, we use 
the nonnegative function 


IG(f)P 
| low ag 


in which the denominator applies the correct normalization in the sense that the integrated 
value of this ratio over the entire frequency axis is unity. We may thus formally define the 
rms bandwidth of a low-pass signal g(t) with Fourier transform G(f) as follows: 
7 v2 
[lew ap 
Warns = | = (A2.25) 


Flew af 
An attractive feature of the rms bandwidth W,,,, is that it lends itself more readily to 


mathematical evaluation than the other two definitions of bandwidth, but it is not as easily 
measurable in the laboratory. 


a TimME-BANDWIDTH PRODUCT 


For any family of pulse signals that differ by a time-scaling factor, the product of the 
signal’s duration and its bandwidth is always a constant, as shown by 


(duration X bandwidth) = constant 
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The product is called the time-bandwidth product or bandwidth-duration product. The 
constancy of the time-bandwidth product is another manifestation of the inverse relation- 
ship that exists between the time-domain and frequency-domain descriptions of a signal, 
In particular, if the duration of a pulse signal is decreased by reducing the time scale by a 
factor a, the frequency scale of the signal’s spectrum, and therefore the bandwidth of the 
signal, is increased by the same factor a, by virtue of the time-scaling property of the Fourier 
transform, and the time-bandwidth product of the signal is thereby maintained constant; 
see item 2 of Table A6.2. For example, a rectangular pulse of duration T seconds has a 
bandwidth (defined on the basis of the positive-frequency part of the main lobe) equal to 
1/T hertz, making the time-bandwidth product of the pulse equal unity. Whatever defini- 
tion we use for the bandwidth of a signal, the time-bandwidth product remains constant 
over certain classes of pulse signals. The choice of a particular definition for bandwidth 
merely changes the value of the constant. 

To be more specific, consider the rms bandwidth defined in Equation (A2.25). The 
corresponding definition for the rms duration of the signal g(t) is 

1/2 


| t? |g(t)|? dt 
Tas = | (A2.26) 


f oa 
where it is assumed that the signal g(ż) is centered around the origin. It may be shown 
that, using the rms definitions of Equations (A2.25) and (A2.26), the time-bandwidth 
product has the following form: 

1 

Tems Wims 2 7 (A2.27) 

4r 

where the constant is 1/47. The Gaussian pulse exp(— mt?) satisfies this condition with the 


equality sign. 


a NOISE EQUIVALENT BANDWIDTH 


The definitions of bandwidth just presented (i.e., 3-dB bandwidth, null-to-null bandwidth, 
and rms bandwidth) are all formulated in terms of deterministic signals. Another definition 
of bandwidth that presents itself in the study of random signals and systems is the noise 
equivalent bandwidth. Suppose that a white noise source of power spectral density No/2 
is connected to the input of the simple RC low-pass filter of Figure A2.1; the corresponding 
value of the average output noise power is equal to No/(4RC). For this filter, the half- 
power or 3-dB bandwidth is equal to 1/(27RC). Here again we find that the average output 
noise power of the filter is proportional to the bandwidth. 

We may generalize this statement to include all kinds of low-pass filters by defining 
a noise equivalent bandwidth as follows. Suppose that we have a source of white noise of 


R 
White | Colored 


noise c noise 


wt) | a(t) 


Figure A2.1 RC low-pass filter. 
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FIGURE A2.2 Illustrating the definition of noise-equivalent bandwidth for a low-pass filter. 


zero mean and power spectral density No/2 connected to the input of an arbitrary low- 
pass filter of transfer function H(f). The resulting average output noise power is therefore 


No {~ z 
Df |Hupyl? af 


= No | THA)? af 


Nout = 
(A2.28) 


where, in the last line, we have made use of the fact that the magnitude response | H(f)| 
is an even function of frequency. 

Consider next the same source of white noise connected to the input of an ideal low- 
pass filter of zero-frequency response H(0) and bandwidth B. In this case, the average 
output noise power is 


Nout = NoBH*(0) (A2.29) 


Therefore, equating this average output noise power to that in Equation (A2.28), we may 
formally define the noise equivalent bandwidth as 


J, bun ag 
B= FF) (A2.29) 
Thus the procedure for calculating the noise equivalent bandwidth consists of replacing 
the arbitrary low-pass filter of transfer function H(f) by an equivalent ideal low-pass filter 
of zero frequency response H(0) and bandwidth B, as illustrated in Figure A2.2. In a similar 
way, we may define a noise equivalent bandwidth for bandpass filters. 


| A2.3 Hilbert Transform 


The Fourier transform is particularly useful for evaluating the frequency content of an 
energy signal or, in a limiting sense, that of a power signal. As such, it provides the math- 
ematical basis for analyzing and designing frequency-selective filters for the separation of 
signals on the basis of their frequency content. Another method of separating signals is 
based on phase selectivity, which uses phase shifts between the pertinent signals to achieve 
the desired separation. The simplest phase shift is that of 180 degrees, which is merely a 
polarity reversal in the case of a sinusoidal signal. Shifting the phase angles of all com- 
ponents of a given signal by 180 degrees requires the use of an ideal transformer. Another 
phase shift of interest is that of +90 degrees. In particular, when the phase angles of all 
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components of a given signal are shifted by +90 degrees, the resulting function of time is 
known as the Hilbert transform of the signal. 

To be specific, consider a signal g(t) with Fourier transform G(f). The Hilbert trans- 
form of g(t), which we shall denote by &(¢), is defined by 


it 
alt) == F a. dr (A2.31) 


Clearly, the Hilbert transformation of g(t) is a linear operation. The inverse Hilbert trans- 
form, by means of which the original signal g(t) is recovered from ĉ(t), is defined by 
g(t) = od (a) dt (A2.32) 
TJ- ti —T 
The functions g(t) and (t) are said to constitute a Hilbert-transform pair. A short table 
of Hilbert-transform pairs is given in Table A6.4. 

We note from the definition of the Hilbert transform that g(t) may be interpreted as 
the convolution of g(t) with the time function 1/at. We also know from the convolution 
theorem that the convolution of two functions in the time domain is transformed into the 
multiplication of their Fourier transforms in the frequency domain; see item 12 of Table 
A6.2. For the time function 1/7t, we have (see Table A6.3) 


: eer 
7 AE j sen(f) (A2.33) 


_ where sgn(f) is the signum function defined in the frequency domain as 


1, f>0 
sgn(f)=7 0, f=0 (A2.34) 
=i; f<0 
It follows therefore that the Fourier transform Ĝ( f) of S(t) is given by 
G(f) = -j sentf)G(f) (A2.35) 


Equation (A2.35) states that given a signal g(t), we may obtain its Hilbert transform 
&(t) by passing g(t) through a linear two-port device whose frequency response is equal to 
—j sgn(f). This device may be considered as one that produces a phase shift of —90 degrees 
for all positive frequencies of the input signal and +90 degrees for all negative frequencies, 
as in Figure A2.3. The amplitudes of all frequency components in the signal, however, are 








FIGURE A2.3 Phase characteristic of linear two-port device for obtaining the Hilbert transform 
of a real-valued signal. 
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unaffected by transmission through the device. Such an ideal device is referred to as a 
Hilbert transformer. 


= PROPERTIES OF THE HILBERT TRANSFORM 


The Hilbert transform differs from the Fourier transform in that it operates exclusively in 
the time domain. It has a number of useful properties, some of which are listed next. The 
signal g(t) is assumed to be real valued, which is the usual domain of application of the 
Hilbert transform. For this class of signals, we may state the following: 


1. A signal g(t) and its Hilbert transform g(t) have the same magnitude spectrum, 
2. If g(t) is the Hilbert transform of g(t), then the Hilbert transform of g(¢) is —g(¢). 


3. A signal g(t) and its Hilbert transform g(t) are orthogonal over the entire time interval 
(—%, %), as shown by 


F g(t)g(t)dt = 0 


Proofs of these properties are left as exercises for the reader; the proofs follow from Equa- 
tions (A2.31), (A2.32) and (A2.35). 


A2.4 Complex Representation 
of Signals and Systems 


E PRE-ENVELOPE 


Consider a real-valued signal g(t). We define the pre-envelope, or analytic signal, of the 
signal g(t) as the complex-valued function 


&+(t) = g(t) + jêlt) (A2.36) 


where g(t) is the Hilbert transform of g(t). We note that the given signal g(t) is the real 
part of the pre-envelope g(t), and the Hilbert transform of the signal is the imaginary 
part of the pre-envelope. Just as the use of phasors simplifies manipulations of alternating 
currents and voltages, so we find that the pre-envelope is particularly useful in handling 
band-pass signals and systems. 

One of the important features of the pre-envelope g.-(£) is the behavior of its Fourier 
transform. Let G.(f) denote the Fourier transform of g,(t). Then we may write 


G.(f) = G(f) + sgen(f)G(f) 
from which we readily find that 
2G(f), f > 0 
Gilf) = 7 G(0), f= 0 (A2.37) 
o f<o0 


where G(0) is the value of G(f) at frequency f = 0. This means that the pre-envelope of 
a signal has no frequency content (i.e., its Fourier transform vanishes) for all negative 
frequencies. 
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From the foregoing analysis it is apparent that for a given signal g(t) we may deter- 
mine its pre-envelope g+ (t) in one of two equivalent ways: 


1. We determine the Hilbert transform g(t) of the signal g(t), and then use Equation 
(A2.36) to compute the pre-envelope g+ (t). 

2. We determine the Fourier transform G(f) of the signal g(t), use Equation (A2.37) to 
determine G+(f), and then evaluate the inverse Fourier transform of G.(f) to obtain 


g(t) = 2 J G(f) exp(j2aft) df (A2.38) 


For a particular signal g(t) of Fourier transform G(f), one of these two ways may be better 
than the other. 

Equation (A2.36) defines the pre-envelope g+ (t) for positive frequencies. Symmetri- 
cally, we may define the pre-envelope for negative frequencies as 


g_(t) = g(t) — 78(2) (A2.39) 


The two pre-envelopes g(t) and g_(t) are simply the complex conjugate of each other, as 
shown by : 


g_(t) = gi(t) (A2.40) 


where the asterisk denotes complex conjugation. The spectrum of the pre-envelope g+(t) 
is nonzero only for positive frequencies, as emphasized in Equation (A2.37); hence, the 
use of a plus sign as the subscript. In contrast, the spectrum of the other pre-envelope g_(t) 
is nonzero only for negative frequencies, as shown by the Fourier transform 


0, f>o 
G_(f) =4G(0), f=0 (A2.41) 
2G(f), f <0 


Thus the pre-envelopes g..(t) and g_(t) constitute a complementary pair of complex-valued 
signals. Note also that the sum of g..(t) and g_(t) is exactly twice the original signal g(t). 


@ CANONICAL REPRESENTATIONS OF BAND-Pass SIGNALS 


Consider a band-pass signal g(t) whose Fourier transform G(f) is nonnegligible only in a 
band of frequencies of total extent 2W, say, centered about some frequency +f,. This is 
illustrated in Figure A2.4a. We refer to f. as the carrier frequency. In the majority of 
communication signals, we find that the bandwidth 2W is small compared with f., and 
so we refer to such a signal as a narrowband signal. However, a precise statement about 
how small the bandwidth must be for the signal to be considered narrowband is not 
necessary for our present discussion. 

Let the pre-envelope of a narrowband signal g(t), with its Fourier transform G(f) 
centered about some frequency +f,, be expressed in the form 


g+(t) = g(t) expl j27f.t) (A2.42) 


We refer to g(t) as the complex envelope of the signal. Equation (A2.42) may be viewed 
as the basis of a definition for the complex envelope g(t) in terms of the pre-envelope g+(t). 
We note that the spectrum of g4 (t) is limited to the frequency band f- W=f=fet W, 
as illustrated in Figure A2.4b. Therefore, applying the frequency-shifting property of the 
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FIGURE A2.4 (a) Magnitude spectrum of band-pass signal g(t). (b) Magnitude spectrum of pre- 
envelope g. (t). (c) Magnitude spectrum of complex envelope g(t). 


Fourier transform to Equation (A2.42), which is described as item 5 in Table A6.2, we 
find that the spectrum of the complex envelope £(t) is limited to the band -W = f =s W 
and centered at the origin as illustrated in Figure A2.4c. That is, the complex envelope 
&(t) of a band-pass signal g(t) is a low-pass signal, which is an important result. 

By definition, the given signal g(t) is the real part of the pre-envelope g+ (t). We may 
thus express the original band-pass signal g(t) in terms of the complex envelope &(t) as 
follows: 


&(t) = Re[(t) exp(j27ft)] (A2.43) 


In general, g(t) is a complex-valued quantity; to emphasize this property, we may express 
it in the form 


&(t) = g(t) + jaolt) (A2.44) 
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where g;(t) and golt) are both real-valued low-pass functions; their low-pass property is 
inherited from the complex envelope Z(t). We may therefore use Equations (A2.43) and 
(A2.44) to express the original band-pass signal g(t) in the canonical, or standard, form; 


g(t) = gilt) cos(2mf.t) — golt) sin(27f.t) (A2.45) 


We refer to g;(t) as the in-phase component of the band-pass signal g(t) and to got) as 
the quadrature component of the signal; this nomenclature recognizes that sin(2 rft) [i.e., 
the multiplying factor of go(t)] is in phase-quadrature with respect to cos(27f,t) [i.e., the 
multiplying factor of g,(t)] and cos(27f-t) is viewed as the reference. 

According to Equation (A2.44), the complex envelope g(t) may be pictured as a time- 
varying phasor positioned at the origin of the (g;, Zo)-plane, as indicated in Figure A2.5a, 
With time ¢ varying, the end of the phasor moves about in the plane. Figure A2.5b shows 
the phasor representation of the complex exponential exp(j27f,t). In the definition given 
in Equation (A2.43), the complex envelope g(t) is multiplied by the complex exponential 
exp( j27f.t). The angles of these two phasors therefore add and their lengths multiply, as 
shown in Figure A2.5c. Moreover, in this latter figure, we show the (gr, go)-plane rotating 
with an angular velocity equal to 277, radians per second. Thus, in the picture portrayed 
here, the phasor representing the complex envelope (t) moves in the (gr, go)-plane and 
at the same time the plane itself rotates about the origin. The original band-pass signal 
g(t) is the projection of this time-varying phasor on a fixed line representing the real axis, 


as indicated in Figure A2.5c. 
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FIGURE A2.5. Illustrating an interpretation of the complex envelope g(t) and its multiplication 
by exp(j27rf.t). 
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Since both g;(t) and go(t) are low-pass signals limited to the band -W = f <= W, 
they may be derived from the band-pass signal g(t) using the scheme shown in Figure 
A2.6a. Both low-pass filters in this figure are identical, each of which has a bandwidth 
equal to W. To reconstruct g(t) from its in-phase and quadrature components, we may 
use the scheme shown in Figure A2.6b. 

The two schemes shown in Figure A2.6 are basic to the study of linear modulation 
systems. The multiplication of the low-pass in-phase component g;(t) by cos(2af.t) and 
the multiplication of the low-pass quadrature component go(t) by sin(27f,t) represent 
linear forms of modulation. Given that the carrier frequency f. is sufficiently large, the 
resulting band-pass function g(t) defined in Equation (A2.45) is referred to as a passband 
signaling waveform. Correspondingly, the mapping from gz(t) and go(t) into g(t) is known 
as passband modulation. 

Equation (A2.44) is the Cartesian form of expressing the complex envelope f(t). 
Alternatively, we may express it in the polar form 


&(t) = a(t) exp[ jo (t)] (A246) 


where a(t) and ¢ (t) are both real-valued low-pass functions. Based on this polar represen- 
tation, the original band-pass signal g(t) is defined by 


g(t) = a(t) cos[2mft + (t)] (A2.47) 


We refer to a(t) as the natural envelope or simply the envelope of the band-pass signal g(t) 
and to $(t) as the phase of the signal. Equation (A2.47) represents a hybrid form of 
amplitude modulation and angle modulation; indeed, it includes amplitude modulation, 
frequency modulation, and phase modulation as special cases. 

From this discussion it is apparent that, whether we represent a band-pass (modu- 
lated) signal g(t) in terms of its in-phase and quadrature components as in Equation 
(A2.45) or in terms of its envelope and phase as in Equation (A2.47), the information 
content of the signal g(t) is completely preserved in the complex envelope §(t). 
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FIGURE A2.6 (a) Scheme for deriving the in-phase and quadrature components of a band-pass 
signal. (b) Scheme for reconstructing the band-pass signal from its in-phase and quadrature 
components. 
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= TERMINOLOGY 


The distinctions among the three different envelopes that we have introduced to describe 
a band-pass signal g(t) should be carefully noted. We summarize their definitions here: 


1. The pre-envelope g+ (t) for positive frequencies is defined by 
glt) = g(t) + igl) 


where ĝ(t) is the Hilbert transform of the signal g(t). According to this representation, 
&(t) may be viewed as the quadrature function of g(t). Correspondingly, in the fre- 
quency domain we have 


2G(f), f > 0 
Gf) =4G(0), f= 0 
0, f<0 


2. The complex envelope g(#) equals a frequency-shifted version of the pre-envelope 
g(t), as shown by : 


E(t) = g(t) exp(—j2mft) 
where f, is the carrier frequency of the band-pass signal g(t). 


3. The envelope a(t) equals the magnitude of the complex envelope (t) and also that 
of the pre-envelope g,(t), as shown by 


a(t) = |g) | = |g-(2)| 


Note that for a band-pass signal g(t), the pre-envelope g,() is a complex band-pass signal 
whose value depends on the carrier frequency f. On the other hand, the envelope a(t) is 
always a real low-pass signal and, in general, the complex envelope g(t) is a complex low- 
pass signal; the values of the latter two envelopes are independent of the choice of the 
carrier frequency f.. This property gives the complex envelope &(t) an analytic advantage 
over the original signal g(t). 

The envelope a(t) and phase $(t) of g(t) are related to the quadrature components 
g(t) and golt) as follows (see the time-varying phasor representation of Figure A2.5a): 


a(t) = Vgi(t) + golt) 
= tan gat) 
ENSE Gs 


Conversely, we may write 


gi(t) = a(t) cosie (t)] 
golt) = a(t) sinle (t)] 
Thus, each of the quadrature components of a band-pass signal contains both amplitude 


and phase information. Both components are required for a unique definition of the phase 
hit), modulo 27. 


a BAND-PASS SYSTEMS 


Now that we know how to handle the complex low-pass representation of band-pass 
signals, it is logical that we develop a corresponding procedure for handling the analysis 
of band-pass systems. Specifically, we wish to show that the analysis of band-pass systems 
can be greatly simplified by establishing an analogy (or, more precisely, an isomorphism) 
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between low-pass and band-pass systems. This analogy is based on the use of the Hilbert 
transform for the representation of band-pass signals. 

Consider a narrowband signal x(t), with its Fourier transform denoted by X(f). We 
assume that the spectrum of the signal x(t) is limited to frequencies within + W Hz of the 
carrier frequency f.. Also, we assume that W < f.. Let this signal be represented in terms 
of its in-phase and quadrature components as follows: 


x(t) = xr(t) cos(2af.t) — xolt) sin(27f,t) (A2.48) 


where x,(¢) is the in-phase component and x0(t) is the quadrature component. Then, using 
x(t) to denote the complex envelope of x(t), we may write 


X(t) = x(t) + jxolt) (A2.49) 


Let the signal x(t) be applied to a linear time-invariant band-pass system with impulse 
response b(t) and frequency response H(f). We assume that the frequency response of the 
system is limited to frequencies within +B of the carrier frequency f.. The system band- 
width 2B is usually narrower than or equal to the input signal bandwidth 2W. We wish 
to represent the band-pass impulse response h(t) in terms of two quadrature components, 
denoted by h;(t) and ho(t). Thus, by analogy to the representation of band-pass signals, 
we may express h(t) in the form 


h(t) = h(t) cos(2rf:t) — holt) sin(27f.t) (A2.50) 
Define the complex impulse response of the band-pass system as 
hit) = hilt) + jholt) (A2.51) 
Hence, we have the complex representation 
h(t) = Re[h(t) expl j27f.t)] (A2.52) 
Note that h;(t), Folt), and h(t) are all low-pass functions limited to the frequency band 
-BsfsB. 
We may determine the complex impulse response 4(t) in terms of the quadrature 
components h;(f) and ho(t) of the band-pass impulse response h(t) by using Equation 


(A2.51). Alternatively, we may determine it from the band-pass frequency response H(f) 
in the following way. We first note from Equation (A2.52) that 


2h(t) = A(t) exp( j2af.t) + h*(t) exp(—j2af.2) (A2.53) 


where h*(2) is the complex conjugate of h(t). Therefore, applying the Fourier transform 
to Equation (A2.53), and using the complex-conjugation property of the Fourier trans- 
form, which is described in item 10 in Table A6.2, we get 


2Hf) = Hf — fà + A*(-f - fà (A2.54) 


where H(f) is the Fourier transform of h(t), and H(f) is the Fourier transform of h(t). 
Equation (A2.54) satisfies the requirement that H* (f) = H(— f) for a real impulse response 
h(t). Since H(f) represents a low-pass frequency response limited to | f| = B with B < f, 
we deduce from Equation (A2.54) that 


if - f) =2Hif), f>0 (A2.55) 


Equation (A2.55) indicates that for a specified band-pass frequency response H(f), we 
may determine H(f) by taking the part of H(f) corresponding to positive frequencies, 
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shifting it to the origin and then scaling it by the factor 2. To determine the complex 
impulse response h(t), we take the inverse Fourier transform of H(f), obtaining 


bi) = | EP) explidefe af (A2.56) 


The representations just described for band-pass signals and systems provide the basis 
of an efficient method for determining the output of a band-pass system driven by a band- 
pass signal. We assume that the spectrum of the input signal x(t) and the frequency re- 
sponse H(f) of the system are both centered around the same frequency f,. In practice, 
there is no need to consider a situation in which the carrier frequency of the input signal 
is not aligned with the midband frequency of the band-pass system, since we have consid- 
erable freedom in choosing the carrier or midband frequency. Thus, changing the carrier 
frequency of the input signal by an amount Af, say, simply corresponds to absorbing (or 
removing) the factor exp(+j27 Aft) in the complex envelope of the input signal or the 
complex impulse response of the band-pass system. We are therefore justified in proceeding 
on the assumption that X(f) and H(f) are both centered around fe. Suppose then we use 
y(t) to denote the output signal of the system. It is clear that y(¢) is also a band-pass signal, 
so that we may represent it in terms of its low-pass complex envelope ¥(t), as follows: 


y(t) = Re[F(t) exp(j27f,0)] (A2.57) 
The output signal y(t) is related to the input signal x(t) and impulse response h(t) of 
the system in the usual way by the convolution integral 


y(t) = i h(t)x(t — 7) dr (A2.58) 


In terms of pre-envelopes, we have h(t) = Re[h-(z)] and x(t) = Re[x.(¢)]. We may therefore 
rewrite Equation (A2.58) in terms of the pre-envelopes x(t) and h.,(¢) as follows: 


y(t) = ia Re[h,(7)] Relx.(t — 7)] dr (A2.59) 


To proceed further, we make use of a basic property of pre-envelopes that is described by 
the following relation (presented here without proof): 


f Re[h.(7)] Refx+(7)] dr = Ere f h.(r)x (7) a (A2.60) 


where we have used 7 as the integration variable to be consistent with that in Equation 
(A2.59). Next, we note that using x(—7) in place of x(t) has the effect of removing the 
complex conjugation on the right-hand side of Equation (A2.60). Hence, bearing in mind 
the algebraic difference between the argument of x,(z) in Equation (A2.60) and that of 
x(t — 7) in Equation (A2.59), and using the relationship between the pre-envelope and 
complex envelope of a band-pass function, we get 


y(t) = Rel f h,(t)x4(t — 7) ar| 


rel f bin) exp(j2af.nx(t — r) exp(j2af.{t — 7)) a| (A2.61) 


Nje MIR 


Ref exp j2nfat) f hinžit — 7) ar 
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Thus comparing the right-hand sides of Equations (A2.57) and (A2.61), we readily deduce 
that for a large enough carrier frequency ££, the complex envelope #(t) of the output signal 
is related to the complex envelope X(f) of the input signal and the complex impulse response 
h(t) of the band-pass system as follows: 


pæ 


25(t) = te A(s)z(t — 7) dr (A2.62) 


or, using the shorthand notation for convolution, 
25t) = b(t) x H(t) (A2.63) 


where * denotes convolution. In other words, except for the scaling factor 2, the complex 
envelope ¥(t) of the output signal of a band-pass system is obtained by convolving the 
complex impulse response h(t) of the system with the complex envelope X(t) of the input 
band-pass signal. Equation (A2.63) is the result of the isomorphism, for convolution, 
between a band-pass function and the corresponding low-pass function. 

The significance of this result is that in dealing with band-pass signals and systems, 
we need only concern ourselves with the low-pass functions £(t), 9(t), and A(t), representing 
the excitation, the response, and the system, respectively. That is, the analysis of a band- 
pass system, which is complicated by the presence of the multiplying factor exp( j27f,t), 
is replaced by an equivalent but much simpler low-pass analysis that completely retains 
the essence of the filtering process. This procedure is illustrated schematically in Figure 
A2.7. 

The complex envelope £(t) of the input band-pass signal and the complex impulse 
response h(t) of the band-pass system are defined in terms of their respective in-phase and 
quadrature components by Equations (A2.49) and (A2.51), respectively. Substituting these 
relations in Equation (A2.63), we get 


25(t) = [hz(t) + jho(t)] & [xi(t) + jxolt)] (A2.64) 


Because convolution is distributive, we may rewrite Equation (A2.64) in the equivalent 
form 


29(t) = [bi(t) X x(t) — holt) X xo(t)] + jiholt) X xilt) + hr X xp(t)] (42.65) 


Let the complex envelope (t) of the response be defined in terms of its in-phase and 
quadrature components as 


H(t) = yilt) + jyol(t) (A2.66) 


Comparing the real and imaginary parts in Equations (A2.65) and (A2.66), we have for 
the in-phase component y;(t) the relation 


























2yr(t) = y(t) X x(t) — holt) X xolt) (A2.67) 
x(f) = Re GW) exp (f27f.0) yd) = Re F(A exp G2rf d] Ed) Z 29) 
o—_—_—S—S_ J ff ————“€ oS K) m> 
(a) {b} 


FIGURE A2.7 (a) Narrowband filter of impulse response h(t) with narrowband input signal x(t). 
(b) Equivalent low-pass filter of complex impulse response h(t) with complex low-pass input <(t). 
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FIGURE A2.8 Block diagram illustrating the relationships between the in-phase and quadrature 
components of the response of a band-pass filter and those of the input signal. 


and for the quadrature component yq(¢) the relation 
2yolt) = Bolt) X x(t) + hilt) X xolt) - (A2.68) 


Thus, for the purpose of evaluating the in-phase and quadrature components of the com- 
plex envelope ÿ(ż) of the system output, we may use the low-pass equivalent model shown 
in Figure A2.8. All the signals and impulse responses shown in this model are real-valued 
low-pass functions. Accordingly, this equivalent model provides a practical basis for the 
efficient simulation of band-pass filters or communication channels on a digital computer. 

To sum up, the procedure for evaluating the response of a band-pass system (with 
mid-band frequency f.) to an input band-pass signal (of carrier frequency f.) is as follows: 


1. The input band-pass signal x(¢) is replaced by its complex envelope *(t), which is 
related to x(t) by 


x(t) = Re[X(t) exp( j27f.t)] 
2. The band-pass system, with impulse response h(t), is replaced by a low-pass analog, 
which is characterized by a complex impulse response h(t) related to h(t) by 
h(t) = Refh(t) exp( j27f.2)| 
3. The complex envelope #(¢) of the output band-pass signal y(t) is obtained by con- 
volving h(t) with £(t), as shown by 
25(t) = A(t) x X(t) 


4, The desired output y(t) is finally derived from the complex envelope $(t) by using 
the relation 


y(t) = Re[F(t) exp(j27f.t)] 





BESSEL FUNCTIONS 


| A3.1 Series Solution of Bessel’s Equation 


In its most basic form, Bessel’s equation of order n is written as 
dy dy í 
2 2 P)\y = s 
x z+ x7 + (x n*)y = 0 (A3.1) 


which is one of the most important of all variable-coefficient differential equations.’ For 
each #, a solution of this equation is defined by the power series 


m+lm 
1 
= (— ri =) 


Fx) = > (43.2) 


mo mi(n + m)! 








The function J,,(x) is called a Bessel function of the first kind of order n. Equation (A3.1) 
has two coefficient functions, namely, 1/x and (1 — 7/x*). Hence, it has no finite singular 
points except the origin. It follows therefore that the series expansion of Equation (A3.2) 
converges for all x > 0. Equation (A3.2) may thus be used to numerically calculate J,,(x) 
forn = 0,1,2,.... Table A6.5 presents values of J,,(x) for different orders # and varying 
x. It is of interest to note that the graphs of Jo{x) and J,(x) resemble the graphs of cos x 
and sin x, respectively; see the graphs of Figure 2.23 in Chapter 2. 
The function J„(x) may also be expressed in the form of an integral as 


Jax) = 1f cos(x sin @ — n8) dé (A3.3) 
or, equivalently, 


1 {* ; : 
Jax) = if exp( jx sin 8 — jn) d0 (A3.4) 


| A3.2 Properties of the Bessel Function 








The Bessel function J„(x) has the following properties: 
1. Jax) = (-1)"J-alx) (A3.5) 
To prove this relation, we replace 6 by (a — 6) in Equation (A3.3). Then, noting that 
sin(az — 6) = sin 0, we get 


Jx) = 1 Í cos{x sin 8 + nê — nm) dé 
T 0 


= 4 Í [cos(nm) cos(x sin @ + n8) + sin(ar) sin(x sin 8 + 16)] dé 
mdo 
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For integer values of n, we have 


cos(ma) = (—1)* 





sin(n7) = 0 
Therefore, 
J = f cos(x sin 0 + n8) dê (A3.6) 
T 0 


From Equation (A3.3), we also find that by replacing n with ~n: 
J_.{x) = if cos(x sin 8 + n8) dé (A3.7) 


The desired result follows immediately from Equations (A3.6) and (A3.7). 
Jn(x) = (—1)"Jn(—2) (A3.8} 


This relation is obtained by replacing x with —x in Equation (A3.3), and then using 
Equation (A.3.6). 


Jal) + Jail) = = Ja) (A309) 


This recurrence formula is useful in constructing tables of Bessel coefficients; its der- 
ivation follows from the power series of Equation (A3.2). 


. For small values of x, we have 


n 


Jax) = Fal (A3.10) 


This relation is obtained simply by retaining the first term in the power series of 
Equation (A3.2) and ignoring the higher-order terms. Thus, when x is small, we have 


Jox) = 1 
hix) = 


T(x) = 0 forn > 1 


(A3.11) 


NIX 


. For large values of x, we have 





Jd) = a cos(x A za) (A3.12) 


This shows that for large values of x, the Bessel function J„(x) behaves like a sine 
wave with progressively decreasing amplitude. 


. With x real and fixed, J,,(x) approaches zero as the order 1 goes to infinity. 


> I(x) exp(ind) = expl jx sin 6) (A313) 


To prove this property, consider the sum Xz J,,(x) expl jn) and use the formula 
of Equation (A3.4) for J,,(x) to obtain 


E Jule) expl ind) = -5 exp( jng) f exp(jx sin @ — jnd) dê 


n= 0 
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Interchanging the order of integration and summation: 


» æ 


È Jia) explind) = 2 | doexplix sino) $ explin(d ~ 0] (A3.14) 


n=% n=—@ 


We now invoke the following relation from Fourier transform theory: 


7 
Sd => Dd explin(d)], --7sos0 (A3.15) 


where ô(¢) is a delta function. Therefore, using Equation (A3.15) in (A3.14) and 
then applying the sifting property of the delta function, we get 


DS J.Ax) exp(jnd) = Y expl jx sin 0) &(@ — 6) do 
es expl jx sin $) 


which is the desired result. 


8. X J4x)=1  foralx (A3.16) 
To prove this property, we may proceed as follows. We observe that J,(x) is real. 
Hence, multiplying Equation (A3.4) by its own complex conjugate and summing 
over all possible values of n, we get 


5 Tx) = 1 5 i: ie exp(jx sin @ — jn0 — jx sind + jnd) do dd 


n=— o (2r) n=- 


Interchanging the order of double integration and summation: 


È Jax) = 
T a ca a (A3.17) 
Oak l B 46 dé exp|jx(sin @ — sind)] > expljn(d — o 
Using Equation (A3.15) in (A3.17) and then applying the sifting property of the delta 


function, we finally get 


: T 
` e= | a1 


Jare 
which is the desired result. 


Many of these properties of the Bessel function J„(x) may also be illustrated in nu- 
merical terms by referring to Table A6.5. 


The modified Bessel equation of order n is written as 


yA 
i 5 +x 2 - (x? + ny = 0 (A3.18) 


x 
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With 7 = —1, where j is the square root of —1, we may rewrite this equation as 
de 
x? me +x T + (fx? — m)y = 0 


From this rewrite it is evident that Equation (A3.18) is nothing but Bessel’s equation, 
namely, Equation (A3.1), with x replaced by jx. Thus replacing x by jx in Equation (A3.2), 


we get 
se n+2m 
E (-1"(2) 
=> 


mo m'(n + m)! 


=f Deo G) 


m=0 o m\(n + m)! 





Next we note that J„( jx) multiplied by a constant will still be a solution of Bessel’s equa- 
tion. Accordingly, we multiply J„( jx) by the constant j~”, obtaining 


( 1 y 
>x 

= 2 
jli) = > alia £ mji 
This new function is called the modified Bessel function of the first kind of order n, denoted 
by I„(x). We may thus formally express a solution of the modified Bessel equation, Equa- 


tion (A3.18), as 


I, aay a n j: 
(x) = j” a on (A319) 
(3+) 


m=o mi(n + m)! 


The modified Bessel function I,,(x) is a monotonically increasing real function of the ar- 
gument x for all n, as shown in Figure A3.1 for n = 0, 1, 2. 


nN 
3 
T 
3 
i 
N 


= 
© 
T 


Modified Besse! Function /,(x) 








0 1 2 3 4 5 6 
x 


FIGURE A3.1 Modified Bessel function I„(x) of varying order n. 
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The modified Bessel function I,,(x) is identical to the original Bessel function J,,(x) 
except for an important difference: The terms in the series expansion of Equation (A3.19) 
are all positive, whereas they alternate in sign in the series expansion of Equation (A3.2). 
The relationship between J„(x) and I,,(x) is analogous to the way in which the trigonometric 
functions cos x and sin x are related to the hyperbolic functions cosh x and sinh x. 

An interesting property of the modified Bessel function I,,(x) is derived from Equation 
(A3.13). Specifically, replacing x by jx and the angle ¢ by @ — 7/2 in this equation, and 
then invoking the definition of I,,(x) in the first line of Equation (A3.19), we obtain 


C3 


5, I(x) exp(jn@) = exp(x cos 0) (43.20) 


From this relation it follows that 
4f 
2m d-r 


L(x) exp(x cos 6) cos(#6) d0 (A3.21) 


This integral formula for I,,(x) may, of course, also be derived from Equation (A3.4) by 
making the appropriate changes. 

When the argument x is small, we obtain the following asymptotic estimates directly 
from the series representation of Equation (A3.19): 


Idx) > 1 for x > 0 (A3.22) 
and 
L(x) > 0 for n = 1 and x > 0 (43.23) 


For large values of x we have the following asymptotic estimate for I,,(x), which is valid 
for all integers n = 0: 





I, (x) = ee for x > © (A3.24) 
Note that this asymptotic behavior of I,,(x) is independent of the order n for large values 


of x. 


i NOTES AND REFERENCES 


1. Equation (A3.1) is named for the German mathematician and astronomer Friedrich Wilhelm 
Bessel (1784-1846). For detailed treatments of the solution to this equation and related 
issues, see Wylie and Barrett (1982) and Watson (1966). 








CONFLUENT 
HYPERGEOMETRIC 
FUNCTIONS 


| A4.1 Kummer's Equation 


The confluent hypergeometric function' is a solution of Kummer’s differential equation: 


dy 
x dx? 


dy 
dx 





+ (b — x) ay=0 ` (A41) 


where, in general, the parameters a and b are complex numbers. For the case when 
b #0, —1, —2, . . . , the solution of Kummer’s equation is defined by the series 


ax  ala+1) x? 


Aila; Dealt a ie ey OF 


avast (A4.2) 


where ‚F(a; b; x) denotes a confluent hypergeometric function parameterized by a and b. 
In this notation, the first subscript denotes the number of factorials in the numerator of 
the general term in Equation (A4.2), the second subscript denotes the number of factorials, 
apart from n!, in the denominator. In Equation (A4.2), both subscripts are clearly 1. 


A4.2 Properties of the Confluent 
Hypergeometric Function 
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Property 1 


For small values of x, the confluent hypergeometric function approximates as 
1F,(a; b; x) = 1+ vi for x > 0 (A4.3) 
This property follows directly from the series expansion of Equation (A4.2). 


Property 2 
For a = —1 and b = 1 we have the exact identity: 
4F,(-1; 1; x) =1-—<x for all x (A4.4) 


This property also follows directly from the series expansion of Equation (A4.2). 
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Property 3 


The confluent hypergeometric function for a = —1/2 and b = 1 is related exactly to the 
modified Bessel function for all x as follows: 


Alps) -on(-S)(a an) +m) aes 


where I(x) is the modified Bessel function of order n. - 


A special case of Equation (A4.5) occurs when x is large. From the definition of the 
modified Bessel function given in Appendix 3, we have the following asymptotic formula 
for large x: 


Ras 


poera. orras (44.6) 
2ax 





j 


Hence, combining Equations (A4.5) and (A4.6), we obtain the simple result 


(~3 1; -x) = af for x > % (A4.7) 
2 T 


| NOTES AND REFERENCES 


1. For a discussion of confluent hypergeometric functions, see Jeffreys and Jeffreys (1956). 
Tabulated values of these functions are presented in Abramowitz and Stegun (1965). 





CRYPTOGRAPHY 


A5.1 


Secrecy is certainly important to the security or integrity of information transmission, 
Indeed, the need for secure communications is more profound than ever, recognizing that 
the conduct of much of our commerce, business, and personal matters is being carried out 
today through the medium of computers, which has replaced the traditional medium of 
papers. 

Cryptology is the umbrella term used to describe the science of secret communica- 
tions; it is derived from the Greek kryptos and logos which mean “hidden” and “word,” 
respectively.t The subject matter of cryptology may be partitioned neatly into cryptogra- 
phy and cryptanalysis. Cryptography deals with the transformations of a message into 
coded form by encryption and the recovery of the original message by decryption, The 
original message to be encrypted (enciphered) is called the plaintext, and the result pro- 
duced by encryption is called a cryptogram or ciphertext; the latter two terms are used 
interchangeably. The set of data transformations used to do the encryption is called a 
cipher; normally, the transformations are parameterized by one or more keys. Cryptanal- 
ysis, on the other hand, deals with how to undo cryptographic communications by break- 
ing a cipher or forging coded signals that may be accepted as genuine. 

Cryptographic systems offer three important services: 


1. Secrecy, which refers to the denial of access to information by unauthorized users. 
2. Authenticity, which refers to the validation of the source of a message. 


3. Integrity, which refers to the assurance that a message was not modified by accidental 
or deliberate means in transit. 


A conventional cryptographic system relies on the use of a single piece of private and 
necessarily secret information known as the key; hence, conventional cryptography is re- 
ferred to as single-key cryptography or secret-key cryptography.” This form of cryptog- 
raphy operates on the premise that the key is known to the encrypter (sender) and by the 
decrypter (receiver) but to no others; the assumption is that once the message is encrypted, 
it is (probably) impossible to do the decryption without knowledge of the key. 

Public-key cryptography,’ also called two-key cryptography, differs from conven- 
tional cryptography in that there is no longer a single secret key shared by two users. 
Rather, each user is provided with key material of one’s own, and the key material is 
divided into two portions: a public component and a private component. The public com- 
ponent generates a public transformation, and the private component generates’a private 
transformation. But, of course, the private transformation must be kept secret for secure 
communication between the two users. 
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Basically, the flow of information in a secret-key cryptographic system is as shown in 
Figure A5.1. The message source generates a plaintext message, which is encrypted into a 
cryptogram at the transmitting end of the system. The cryptogram is sent to an authorized 
user at the receiving end over an “insecure” channel; a channel is considered insecure if 


Enemy 
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FIGURE A5.1 Block diagram of secret-key cryptographic system. 
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its security is inadequate for the needs of its users. It is assumed that in the course of 
transmission the cryptogram may be intercepted by an enemy cryptanalyst’ (i.e., would- 
be intruder into a cryptographic system). The requirement is to do the encryption in such 
a way that the enemy is prevented from learning the contents of the plaintext message. 

In abstract terms, a cryptographic system or cipher (for short) is defined as a set of 
invertible transformations of the plaintext space (i.e., the set of possible plaintext messages) 
into the cryptogram space (i.e., the set of all possible cryptograms). Each particular trans- 
formation corresponds to encryption (enciphering) of a plaintext with a particular key. 
The invertibility of the transformation means that unique decryption (deciphering) of the 
cryptogram is possible when the key is known. Let X denote the plaintext message, Y 
denote the cryptogram, and Z denote the key. Let F denote the invertible transformation 
producing the cryptogram Y, as follows: 


Y = F(X, Z) = F,(X) (A5.1) 


The transformation is intended to make the cryptogram Y useless to the enemy. At the 
receiving end of the system, the cryptogram Y is decrypted with the inverse transformation 
F~! to recover the original plaintext message X, as shown by 


F(Y, Z) = F7“Y¥) = F7 "(F.(X)) = X (A5.2) 


In physical terms, the cryptographic system consists of a set of instructions, a piece 
of physical hardware, or a computer program. In any event, the system is designed to have 
the capability of encrypting the plaintext (and, of course, decrypting the resulting cryp- 
togram) in a variety of ways; the particular way chosen to do the actual encryption is 
determined by the specific key. 

The security of the system resides in the secret nature of the key, which requires that 
the key must be delivered to the receiver over a secure channel (e.g., registered mail, courier 
service) as implied in Figure A5.1. The cryptographic system depicted in this figure provides 
a solution to the secrecy problem, preventing an enemy from extracting information from 
messages transmitted over an insecure communication channel. Cryptography also pro- 
vides a solution to the authentication problem, preventing an enemy cryptanalyst from 
impersonating the message sender. In this second situation, the enemy cryptanalyst is the 
one who originates a “fraudulent” cryptogram Y’ that is delivered to the receiver (decryp- 
ter), as shown in Figure A5.2. The authentic cryptogram Y is shown as a dashed input to 
the enemy cryptanalyst, indicating that the enemy produces the fraudulent cryptogram Y’ 
without ever seeing the authentic one. The receiver may be able to recognize Y’ as fraud- 
ulent by decrypting it with the correct key Z; hence, the line from the receiver output to 
the destination is shown dashed to suggest rejection of the fraudulent cryptogram Y’ by 
the receiving user. 
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FIGURE A5.2 Illustrating the intrusion of an enemy cryptanalyst. 


| A5.2 Block and Stream Ciphers 


Much as error-correcting codes are classified into block codes and convolutional codes, 
cryptographic systems (ciphers) may be classified into two broad classes: block ciphers and 
stream ciphers. Block ciphers operate in a purely combinatorial fashion on large blocks of 
plaintext, whereas stream ciphers process the plaintext in small pieces (i.e., characters or 
bits). . 

Figure A5.3 shows the generic form of a block cipher. The plaintext (consisting of 
serial data) is divided into large blocks, each of which is usually made up of a fixed number 
of bits. Successive blocks of the plaintext are enciphered (encrypted) using the same secret 
key, otherwise independently; the resulting enciphered blocks are finally converted into 
serial form. Thus, a particular plaintext block identical to a previous such block gives rise 
to an identical ciphertext block. Specifically, each bit of a particular ciphered block is 
chosen to be a function of all the bits of the associated plaintext block and the key; the 
goal of a block cipher is to have no specific bit of the plaintext ever appear in the ciphertext 
directly. 

Block ciphers operate with a fixed transformation applied to large blocks of plaintext 
data, on a block-by-block basis. In contrast, a stream cipher operates on the basis of a 
time-varying transformation applied to individual bits of the plaintext. The most popular 
stream ciphers are the so-called binary additive stream ciphers, the generic form of which 
is shown in Figure AS.4. In such a cipher, the secret key is used to control a keystream 
generator that emits a binary sequence called the keystream, whose length is much larger 
than that of the key. Let x, y,, and z, denote the plaintext bit, ciphertext bit, and key- 
stream bit at time 2, respectively. The ciphertext bits are then determined by simple mod- 
ulo-2 addition of the plaintext bits and the keystream bits, as shown by 


Yn = Xn D Zm n=1,2,...,N (A5.3) 


where N is the length of the keystream. Because addition and subtraction in modulo-2 
arithmetic are exactly the same, Equation (A5.3) also implies the following relation 


Xn = Yn Ð Zw n=1,2,...,N (A5.4) 
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FIGURE A5.3 Block diagram of a block cipher. 
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FIGURE A5.4 Binary additive stream cipher. 


We thus see that in binary additive stream ciphers, identical devices can be used to perform 
encryption and decryption, as shown in Figure A5.4. The secret key is chosen according 
to some probability distribution. To provide secure encryption, the keystream should re- 
semble a coin-tossing (i.e., completely random) sequence as closely as possible. 

Block ciphers are normally designed in such a way that a small change in an input 
block of plaintext produces a major change in the resulting output. This error propagation 
property of block ciphers is valuable in authentication in that it makes it improbable for 
an enemy cryptanalyst to modify encrypted data, unless knowledge of the key is available. 
On the other hand, a binary additive stream cipher has no error propagation; the decryp- 
tion of a distorted bit in the ciphertext affects only the corresponding bit of the resulting 
output. 

Stream ciphers are generally better suited for the secure transmission of data over 
error-prone communication channels; they are used in applications where high data rates 
are a requirement (as in secure video, for example) or when a minimal transmission delay 
is essential,” 


i REQUIREMENT FOR SECRECY 


In cryptography, a fundamental assumption is that an enemy cryptanalyst has knowledge 
of the entire mechanism used to perform encryption, except for the secret key. We may 
identify the following forms of attack that may be attempted by the enemy cryptanalyst, 
depending on the availability of additional knowledge: 


1. Ciphertext-only attack is a cryptanalytic attack in which the enemy cryptanalyst has 
access to part or all of the ciphertext. 

2. Known-plaintext attack is a cryptanalytic attack in which the enemy cryptanalyst 
has knowledge of some ciphertext-plaintext pairs formed with the actual secret key. 

3. Chosen-plaintext attack is a cryptanalytic attack in which the enemy cryptanalyst is 
able to submit any chosen plaintext message and receive in return the correct 
ciphertext for the actual secret key. i 

4. Chosen-ciphertext attack is a cryptanalytic attack in which the enemy cryptanalyst 
is able to choose an arbitrary ciphertext and find the correct result for its decryption. 


A ciphertext-only attack occurs frequently in practice. In this form of attack, an 
enemy cryptanalyst uses only knowledge of the statistical structure of the language in use 
(e.g., in English the letter e occurs with a probability of 13 percent, and the letter q is 
always followed by #) and knowledge of some probable words (e.g., a letter probably 
begins with “Dear Sir/Madam:”). A known-plaintext attack may take place by virtue of 
the standard computer formats used in programming languages and data generation. In 
any case, the ciphertext-only attack is viewed as the weakest threat to which a crypto- 
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graphic system can be subjected, and any system that succumbs to it is therefore considered 
totally insecure. Thus, for a cryptographic system to provide secrecy, at the minimum it 
should be immune to ciphertext-only attacks; ideally, it should also be immune to known- 
plaintext attacks. 


| A5.3  Information-Theoretic Approach 


In the Shannon model of cryptography, named in recognition of Shannon’s 1949 landmark 
paper on the information-theoretic approach to secrecy systems, the enemy cryptanalyst 
is assumed to have unlimited time and computing power. But the enemy is presumably 
restricted to a ciphertext-only attack. Cryptanalysis in the Shannon model is defined as the 
process of finding the secret key, given the cryptogram (ciphertext) and the a priori prob- 
abilities of the various plaintexts and keys. The secrecy of the system is considered broken 
when the enemy cryptanalyst performs decryption successfully, obtaining a unique solution 
to the cryptogram.° 

Let X = (X4, X2,..., Xx) denote an N-bit plaintext message, and Y = (Yj, Yo,..., 
Yy) denote the corresponding N-bit cryptogram; that is, both the plaintext and the cryp- 
togram have the same number of bits. It is assumed that the secret key Z used to construct 
the cryptogram is drawn according to some probability distribution. The uncertainty about 
X is expressed by the entropy H(X), and the uncertainty about X given knowledge of Y 
is expressed by the conditional entropy H(X |Y). The mutual information between X and 
Y is defined by 


KX; Y) = H(X) — H(X|Y) (AS.5) 


The mutual information I(X; Y) represents a basic measure of security (secrecy) in the 
Shannon model. 


æ PERFECT SECURITY 


Assuming that an enemy cryptanalyst can observe only the cryptogram Y, it seems appro- 
priate that we define the perfect security of a cryptographic system to mean that the plain- 
text X and the cryptogram Y are statistically independent. In other words, we have 


U(X; Y) = 0 (A5.6) 

Then, using Equation (A5.5), we find that the condition for perfect security may be re- 
written as 

H(X|Y) = H(X) (AS.7) 


Equation (A5.7) states that the best an enemy cryptanalyst can do, given the cryptogram 
Y, is to guess the plaintext message X according to the probability distribution of all 
possible messages. 

Given the secret key Z, we recognize that 


H(X|Y) = H(X, Z|Y) 
= H(Z|Y) + H(X|Y, Z) 
The conditional entropy H(X|Y, Z) is zero if, and only if, Y and- Z together uniquely 
determine X; this is indeed a valid assumption when the decryption process is performed 
with knowledge of the secret key Z. Hence, we may simplify Equation (A5.8) as follows: 
H(X|Y) = H(Z|Y) 
= H(Z) 


(A5.8) 


(A5.9) 
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Thus, substituting Equation (A5.9) into (A5.7), we find that for a cryptographic system to 
provide perfect security, the following condition must be satisfied: 


H(Z) = H(X) (A5.10) 


The inequality of Equation (A5.10) is Shannon’s fundamental bound for perfect security; 
it states that for perfect security, the uncertainty of a secret key Z must be at least as large 
as the uncertainty of the plaintext X that is concealed by the key. 

For the case when the plaintext and key alphabets are of the same size, the use of 
Shannon’s bound for perfect security yields the following result: The key must be at least 
as long as the plaintext. The conclusion to be drawn from this result is that the length of 
the secret key needed to build a perfectly secure cryptographic system may be impractically 
large for most applications. Nevertheless, perfect security has a place in the practical pic- 
ture: It may be used when the number of possible messages is small or in cases where the 
greatest importance is attached to perfect security. 

A well-known, perfectly secure cipher is the one-time pad’ (sometimes called the 
Vernam cipher), which is used for unconventional applications such as two users com- 
municating on a hotline with high confidentiality requirements. The one-time pad is a 
stream cipher for which the key is the same as the keystream, as shown in Figure AS.5. 
For encryption the input consists of two components: a message represented by a sequence 
of message bits {x,,|7 = 1, 2,...}, and a key represented by a sequence of statistically 
independent and uniformly distributed bits {z,|n = 1, 2,...}. The resultant cipher 
{y,|2 = 1, 2,...} is obtained by the modulo-2 addition of the two input sequences, as 
shown by 


Yn = Xn D Zm n=1,2,... 


Consider, for example, the binary message sequence 00011010 and the binary key se- 
quence 01101001. The modulo-2 addition of these two sequences is written as follows: 


Message: 00011010 
Key: 01101001 
Cipher: 01110011 


In the encryption rule described here, key bit 1 interchanges Os and 1s in the message 
sequence, and key bit 0 leaves the message bits unchanged. The message sequence is re- 
covered simply by modulo-2 addition of the binary cipher and key sequences, as shown 
by 

Cipher: 01110011 

Key: 01101001 

Message: 00011010 


The one-time pad is perfectly secure, because the mutual information between the message 
and the cipher is zero; it is therefore completely undecipherable. 


Key 
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FIGURE A5.5 One-time pad (Vernam cipher). 


748 


APPENDIX 5 CRYPTOGRAPHY 


m Unicity DISTANCE 


Consider now the practical case of an imperfect cipher and ask the question: When can 
an enemy cryptanalyst break the cipher? As the amount of intercepted text increases, 
intuitively we expect that a point may be reached at which it becomes possible for 
an enemy cryptanalyst with unlimited time and computing power to find the key and 
thus break the cipher. This critical point in the Shannon model is called the unicity dis~ 
tance, which is formally defined as the smallest N such that the conditional entropy 
H(Z|Y,, Y¥2,..., Yy) is approximately zero. For a particular kind of “random cipher,” 
the unicity distance is approximately given by* 


_ AZ) 
~ rilog Ly 





A (45.11) 
where H(Z) is the entropy of the key Z, and L, is the size of the ciphertext alphabet. The 
parameter r is the percentage redundancy of the message information contained in the 
N-bit ciphertext; it is itself defined by 


H(X) 


-NeeL (A5.12) 


r= 


where H(X) is the entropy of the plaintext X. In most cryptographic systems, the size L, 
of the ciphertext alphabet is the same as the size L, of the plaintext alphabet; in such a 
case, r is just the percentage redundancy of the plaintext itself. Although the derivation of 
Equation (A5.11) assumes a certain well-defined “random cipher,” it can be used to esti- 
mate the unicity distance for ordinary types of ciphers, which is the routine practice today. 

Let K be the number of digits in the key Z that are chosen from an alphabet of size 
L,; then we may express the entropy of the key Z as follows: 


H(Z) < log(L¥) = K log L; (A5.13) 


with equality if and only if the key is completely random. Let the size L, of the key alphabet 
be the same as the size L, of the ciphertext alphabet, and let the key be chosen completely 
at random to maximize the unicity distance. Then, substituting Equation (A5.13) with 
equality into Equation (A5.11), we get the simple result 


No=— (A5.14) 


To illustrate the application of Equation (A5.14), consider a cryptographic system with 
L, = L, = La, which is used for the encryption of English text. The percentage redundancy 
r for typical English text is about 75 percent. Hence, according to Equation (A5.14), an 
enemy cryptanalyst can break the cipher after intercepting only about 1.333K bits of 
ciphertext data, where K is the key size. 

However, it is important to note that an imperfect cipher that is potentially breakable 
can still be of practical value. When the intercepted ciphetext contains sufficient infor- 
mation to satisfy Equation (A5.11), there is no guarantee that an enemy cryptanalyst with 
limited computational resources can actually break the cipher. Specifically, it is possible 
for the cipher to be designed in such a way that the task of the cryptanalysis, though 
known to be attainable with a finite amount of computation, is so overwhelming that it 
will literally exhaust the physical computing resources of the universe. In such a case, the 
imperfect cipher is said to be computationally secure. 
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@ ROLE oF Data COMPRESSION IN CRYPTOGRAPHY 


Lossless data compression or data compaction is a useful tool in cryptography. We say 
this because data compaction removes redundancy, thereby increasing the unicity distance 
No in accordance with Equation (A5.11). To exploit this idea, data compaction is used 
prior to encryption in the transmitter, and the redundant information is reinserted after 
decryption in the receiver; the net result is that the authorized user at the receiver output 
sees no difference, and yet the information transmission has been made more secure. It 
would be tempting to consider the use of perfect data compaction to remove all redun- 
dancy, thereby transforming a message source into a completely random source and re- 
sulting in No = © with any key size. Unfortunately, we do not have a device capable of 
performing perfect data compaction on realistic message sources, nor is it likely that there 
will ever be such a device. It is therefore futile to rely on data compaction alone for data 
security. Nevertheless, limited data compaction tends to increase security, which is the 
reason why cryptographers view data compression as a useful trick. 


m DIFFUSION AND CONFUSION 


In the Shannon model of cryptography, two methods suggest themselves as general prin- 
ciples to guide the design of practical ciphers. The methods are called diffusion and con- 
fusion, the aims of which (by themselves or together) are to frustrate a statistical analysis 
of ciphertext by the enemy and therefore make it extremely difficult to break the cipher. 

In the method of diffusion, the statistical structure of the plaintext is hidden by 
spreading out the influence of a single bit in the plaintext over a large number of bits in 
the ciphertext. This spreading has the effect of forcing the enemy to intercept a tremendous 
amount of material for the determination of the statistical structure of the plaintext, since 
the structure is evident only in many blocks, each one of which has a very small probability 
of occurrence, In the method of confusion, the data transformations are designed to com- 
plicate the determination of the way in which the statistics of the ciphertext depend on the 
statistics of the plaintext. Thus, a good cipher uses a combination of diffusion and 
confusion. 

For a cipher to be of practical value, however, it must not only be difficult to break 
the cipher by an enemy cryptanalyst, but also it should be easy to encrypt and decrypt 
data given knowledge of the secret key. We may satisfy these two design objectives using 
a product cipher, based on the notion of “divide and conquer.” Specifically, the imple- 
mentation of a strong cipher is accomplished as a succession of simple component ciphers, 
each of which contributes a modest amount of diffusion and confusion to the overall 
makeup of the cipher. Product ciphers are often built using substitution ciphers and trans- 
position ciphers as basic components; these simple ciphers are described next. 


1. Substitution cipher. 

In a substitution cipher each letter of the plaintext is replaced by a fixed substitute, usually 
also a letter from the same alphabet, with the particular substitution rule being determined 
by the secret key. Thus the plaintext 


X= (xi; X25 X3s Xgy e. .) 
where x1, x2, X3, . - . are the successive letters, is transformed into the ciphertext 


Y = (Yi, Yz Ys; Yas «+ +) 


= (flay), flocs), Fls), Flca), - -) (43.13) 
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Plaintext 

letters ABCDEFGHIJKLMNOPQRSTUVWXYZ 
Ciphertext 

letters YDUBHNACSYXELPFMKQJRWGOZIT 


FIGURE A5.6 Substitution cipher. 


where f(-) is a function with an inverse. When the substitutes are letters, the key is a 
permutation of the alphabet. Consider, for example, the ciphertext alphabet of Figure 
A5.6, where we see that the first letter Y is the substitute for A, the second letter D is the 
substitute for B, and so on. The use of a substitution cipher results in confusion. 


2. Transposition cipher. In a transposition cipher, the plaintext is divided into groups of 
fixed period d and the same permutation is applied to each group, with the particular 
permutation rule being determined by the secret key. For example, consider the permu- 
tation rule described in Figure A5.7, for which the period is d = 4. According to this 
cipher, letter x, is moved from position 1 in the plaintext to position 4 in the ciphertext. 
Thus, the plaintext 


X = (x1, X2, X3, Xay Xs, Xey X7, Xey- -) 
is transformed into the ciphertext 
Y = (X3, X43 X2, Xis X75 Xgs Key Kss. e.) 


Although the single-letter statistics of the ciphertext Y are the same as those of the plaintext 
X, the higher-order statistics are changed. The use of a transposition cipher results in 
diffusion. 


By interleaving the simple substitutions and transpositions and repeating the interleaving 
process many times, it is possible to build a strong cipher equipped with good diffusion 
and confusion. 


$ EXAMPLE A5.1 
Consider the plaintext message 
THE KING IS DEAD LONG LIVE THE KING 


Using the permuted alphabet described.in Figure A5.6 for the substitution cipher, this plaintext 
is transformed into the ciphertext 


RCHXSPASJBHYBEFPAESGHRCHXSPA 


Suppose next we apply the permutation rule described in Figure AS.7 for the transposition 
cipher; accordingly, the ciphertext resulting from the substitution cipher is further transformed 
into 


HXCRASPSHYBJFBEBSGEACHRHPASX 


which has no resemblance to the original plaintext. <4 


Plaintext 
letters a %Q Ay % 
Ciphertext 
eats %3  %4 2 1 


FIGURE A5.7 Transposition cipher. 
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| A5.4 Data Encryption Standard 


The data encryption standard (DES)’ is certainly the best known, and arguably the most 
widely used, secret-key cryptoalgorithm; the term algorithm is used to describe a sequence 
of computations. The basic DES algorithm can be used for both data encryption and data 
authentication. It is the standard cryptoalgorithm for data storage and mail systems, elec- 
tronic funds transfers (retail and wholesale), and electronic business data interchange. 

The DES algorithm is a strong block cipher that operates on 64-bit blocks of plaintext 
data and uses a 56-bit key; it is designed in accordance with Shannon’s methods of dif- 
fusion and confusion. Essentially the same algorithm is used for encryption and decryption. 
The overall transformations employed in the DES algorithm may be written as 
P~‘{F[P(X)]}, where X is the plaintext, P is a certain permutation, and the function F 
combines substitutions and transpositions. The function F is itself obtained by cascading 
a certain function f, with each stage of the cascade referred to as a round. 

The flow-chart of Figure A5.8 shows the details of the DES algorithm for encryption. 
After a certain initial permutation, a plaintext of 64 bits is divided into a left-half L and 
a right-half Ro, each of which is 32 bits long. The algorithm then performs 16 rounds of 
a key-dependent computation, with the ith round of the computation described as follows: 


L= Rii i=1,2,...,16 (A5.16) 
R: = L @ AR- Z) i=1,2,...,16 (A5.17) 


On the right-hand side of Equation (A5.17), the addition is modulo-2 and each Z, is a 
different 48-bit block of the key used in round i. The function f(+, +) is a function with a 
32-bit output. The result of the 16th round is reversed, obtaining the sequence RigLi¢. 
This 32-bit sequence is input into a final permutation P~! to produce the 64-bit ciphertext. 
The aim is that after 16 rounds of key-dependent computations, the patterns in the original 
plaintext are undetectable in the ciphertext. From Equations (A5.16) and (A5 .17), we note 
that for decryption the function f(-, -) need not be invertible, because (L;_1, R,-1) can be 
recovered from (L;, R;) simply as follows; 


Ri-1 = L; i = 1, 2,..., 16 (A5.18) 
Li- = R; ® f(L; Z) i= 1, 2,..., 16 (A5.19) 


Equation (A5.19) holds even if the function f(-, +) is a many-to-one function (i.e., it does 
not have a unique inverse). 

Figure A5.9 shows the flowchart for computing the function f(+, +). The 32-bit block 
R is first expanded into a new 48-bit block R’ by repeating the edge bits of each successive 
4-bit word (i.e., the bits numbered 1, 4, 5, 8, 9, 12, 13, 16,..., 28, 29, 32). Thus, given 
the 32-bit block R written as 


R= rara Vst 67708 ERA 129730731732 
— S - 
first second eighth 
4-bit word 4-bit word 4-bit word 


we construct the expanded 48-bit block R’ as follows: 


R! = ryarqratstat's 14h 516171 sPo we ¥28729730”31”3271 
en eee —__y—_—— 
first second eighth 


6-bit word 6-bit word 6-bit word 
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Figure A5.8 Data encryption standard. (From Diffie and Hellman, 1979, with permission of 
the IEEE.) 
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FIGURE A5.9 f(R, K) flowchart. (From Diffie and Hellman, 1979, with permission of the 
IEEE.) 











The 48-bit blocks R’ and Z; are added modulo-2, and the resultant is divided into eight 
6-bit words. Let these words be denoted by B,, Bz,..., Bg. We thus write 


B,B, ++: Ba = R OZ (A5.20) 


Each 6-bit word B; is input to a substitution box S; in the form of a look-up table, pro- 
ducing a 4-bit output S,(B;). Each output bit of the substitution box S; (B;) is a Boolean 
function of the 6-bit word B;. The eight outputs S,(B;), S2(B2),..., Sg(Bg) are arranged 
into a single 32-bit block that is input to the permutation box denoted by P[+]. The per- 
muted output so produced is the desired 32-bit function f(R, Z,), as shown by 


F(R, Zi) = P[S,(B,)S2(B2) Sere Ss(Bs)] (A5.21) 


The 48-bit block Z; for the ith iteration uses a different subset of the 64-bit key Zo. 
The procedure used to determine each Z; is called the key-schedule calculation, the flow- 
chart of which is shown in Figure A5.10. The key Zo has eight parity bits in positions 8, 
16,..., 64, which are used for error detection in their respective 8-bit bytes; the errors 
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FIGURE A5.10 Flowchart for the key-schedule calculation. (From Diffie and Hellman, 1979, 
with permission of the IEEE.) 


of concern may arise in the generation, distribution, and storage of the key Zo. The per- 
muted choice 1 disregards the parity bits of Zo and then permutes the remaining 56 bits 
that are loaded into two 28-bit shift registers, each with 24 taps. The 48 taps of the two 
shift registers are subjected to 16 iterations of computation, with each iteration involving 
one or two cyclic left shifts followed by a permutation, referred to as permuted choice 2. 
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The outputs resulting from these 16 iterations provide the different 48-bit blocks Z,, 
Z25+.-, Zi¢ Of the key used in iteration 1, 2, . . . , 16, respectively. 

Despite all the claims to the contrary, it appears that no one has yet demonstrated a 
fundamental weakness of the DES algorithm. Notwithstanding all the controversy sur- 
rounding its use, perhaps the most significant contribution of the DES algorithm is the fact 
that it has been instrumental in raising the level of interest in using cryptography as a 
mechanism for secure computer networks. 


| A5.5 Public-Key Cryptography” 


For a pair of users to engage in cryptographic communication over an insecure channel, 
it is necessary for the users to exchange key information prior to communication. The 
requirement for a secure distribution of keys among authorized users applies to all cryp- 
tographic systems, regardless of their type. In conventional cryptography, the users employ 
a physically secure channel (e.g., courier service or registered mail) for key distribution. 
However, the use of such a supplementary channel points to a major limitation of con- 
ventional cryptography. Needless to say, the use of courier service or registered mail for 
key distribution is costly, inconvenient, low-bandwidth, and slow; also, it is not always 
secure. 

The problem of key distribution is particularly accentuated in large communication 
networks, where the number of possible connections grows as (n? — )/2 for n users. For 
large n, the cost of key distribution becomes prohibitive. Thus, in the development of large, 
secure communication networks, we are compelled to rely on the use of insecure channels 
for both exchange of key information and subsequent secure. communication. This con- 
straint raises a fundamental question: How can key information be exchanged securely 
over an insecure channel? In public-key cryptography, this seemingly difficult issue is re- 
solved by making some key material “public” and thereby considerably simplifying the 
task of key management. This is in direct contrast to conventional cryptography, where 
the key is kept completely secret from an enemy cryptanalyst. 

A public key cryptographic system is described by two sets of algorithms that com- 
pute invertible functions (transformations). Let these two sets of algorithms be denoted by 
{E,] and {D,} that are indexed by z. The invertible transformations computed by these 
algorithms may be written as follows 


Ez: f(x) = y (45.22) 
Da fz'(y) = x (A5,23) 


where x is a certain input message in the domain of some function f, indexed by z, and y 
is the corresponding cryptogram in the range of f,. A fundamental requirement of the 
system is that the function f, must be a trapdoor one-way function. The term “one-way” 
refers to the fact that for x in the domain of f,, it must be easy to compute f,(x) from 
knowledge of the algorithm E,, but for a certain cryptogram y in the range of f,, an enemy 
cryptanalyst must find it extremely difficult to compute the inverse f; '(y). On the other 
hand, an authorized user in possession of the associated algorithm D, would find it easy 
to compute the inverse f7 '(y). Thus the private key (algorithm) D, provides a “trapdoor” 
that makes the problem of inverting the function f, appear extremely difficult from the 
viewpoint of the cryptanalyst, but easy for the (sole authorized) possessor of D,. Since 
knowledge of the key (algorithm) E, does not by itself make it possible to compute the 
inverse of f,, it may be made public; hence, the name “public-key cryptography.” 
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The notion emerging from the description of a public-key cryptographic system pre- 
sented herein is that the keys come in inverse pairs (i.e., public key and private key), and 
that each pair of keys has two basic properties: 


1. Whatever message is encrypted with one of the keys can be decrypted with the other 
key. : 

2. Given knowledge of the public key, it is computationally infeasible to find the secret 
key. 


The use of public-key cryptography as described herein makes it possible to solve 
the secrecy problem as follows. Subscribers to a secure communication system list their 
public keys in a “telephone directory” along with their names and addresses. A subscriber 
can then send a private message to another subscriber simply by looking up the public key 
of the addressee and using the key to encrypt the message. The encrypted message (i.e., 
ciphertext) can only be read by the holder of that particular public key. In fact, should the 
original message (i.e., plaintext) be lost, even its sender would find it extremely difficult to 
recover the message from the ciphertext. 

The key management of public-key cryptography makes it well suited for the devel- 
opment of large, secure communication networks. Indeed, it has evolved from a simple 
concept to a mainstay of cryptographic technology. 


æ DirFir—HELLMAN PuBLic KEY DISTRIBUTION. 


In a simple and yet elegant system known as the Diffie-Hellman public key-distribution 
system, use is made of the fact that it is easy to calculate a discrete exponential but difficult 
to calculate a discrete logarithm. To be more specific, consider the discrete exponential 
function 


Y=a*modp frisXxsp-1 (A5.24} 


where the arithmetic is performed modulo-p. The a is an integer that should be primitive 
(i.e. all powers of a generate all the elements mod p relatively prime to p — 1). Corre- 
spondingly, X is referred to as the discrete logarithm of Y to the base a, mod p, as shown 
by f 

X = logeY mdp forl1=Y=p-1 (A5.25) 


The calculation of Y from X is easy, using the trick of square-and-multiply. For example, 
for X = 16 we have 


Y = a" = {[(P PP? 


On the other hand, the problem of calculating X from Y is much more difficult. 

In the Diffie-Hellman public key-distribution system, all users are presumed to know 
both æ and p. A user i, say, selects an independent random number X; uniformly from the 
set of integers (1, 2,..., p} that is kept as a private secret. But the discrete exponential 


Y; = a“ mod p (A5.26) 


is deposited in a public directory with the user’s name and address. Every other user of 
the system does the same thing. Now, suppose that users i and j wish to communicate 
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privately. To proceed, user i fetches Y; from the public directory and uses the private secret 
X; to compute 


K; = (Y)* mod p 
= (o%)* mod p (AS.27) 


= œ" mod p 


In a similar way, user j computes K;. But we have 


K; = K; (A5.28) 
J ay 


Accordingly, users i and j arrive at K; as the secret key in a conventional cryptosystem. 
Another user must compute K; using the information Y; and Y; obtained from the public 
directory, applying the alternative formula 


K; = (Y,)!°8% mod p (A5.29) 


Apparently, there is no other method for an enemy to find the secret key K;;; however, 
there is no proof for it. In light of what we said earlier, Equation (A5.29) is difficult to 
calculate as it involves a discrete logarithm, whereas Equation (A5.27) is easy to calculate 
as it involves a discrete exponential. Thus, security of the system depends on the difficulty 
encountered in computing a discrete logarithm. 


The Diffie-Hellman public key-distribution system is the oldest system in its class; 


nevertheless, it is still generally considered to be one of the most secure and practical public 
key-distribution systems. 


| A5.6 Rivest-Shamir—Adleman System 


To develop a public-key cryptographic system is no easy task. Indeed, numerous such 
systems have been proposed in the literature, but unfortunately most of them have proven 
to be insecure. To date, the most successful implementation of public-key cryptography is 
the Rivest-Shamir-Adleman (RSA) system,'' which uses ideas from classical number the- 
ory. It is considered to be one of the most secure cryptographic systems in that it has 
withstood many attempts by experts in the field to break it. 


The RSA algorithm is a block cipher based on the fact that finding a random prime 


number of large size (e.g., 100 digit) is computationally easy, but factoring the product of 
two such numbers is currently considered computationally infeasible. Specifically, the com- 
putation of parameters specific to the RSA algorithm proceeds as follows: 


1. 


2. 


Choose two very large prime numbers, p and q, at random; the prime numbers have 
to be fairly carefully chosen as some prime numbers lead to a very weak system. 


Multiply the numbers p and q, obtaining the product 
pq=n (A5.30) 
Find the Euler totient function of n, using the formula. 
(2) = (p — 1X4- 1) (AS5.31) 


Equation (A5.31) follows from the definition of the Euler totient function ¢(#) as 
the number of positive integers i less than z, such that the greatest common divisor 
of i and n is equal to one. 
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3. Let e be a positive integer less than ¢ (7), such that the greatest common divisor of 
e and ¢ (7) is equal to one. Hence, find a positive integer d less than ¢ (7), such that 


de = 1 mod ¢({n) (A5.32) 


The RSA trapdoor one-way function is then defined simply by computing the discrete 
exponentiation 


fAx) = x? = y mod n (A5.32) 


The values of n and e constitute the public key; hence, publishing the easy-to-find algorithm 
E, to compute the function f, amounts just to publishing the numbers # and e. 

The prime numbers p and q constitute the private key. Since d is related to p and q, 
possession of the easy-to-find (when one knows the trapdoor z) algorithm D, to compute 
the inverse function f;! amounts just to knowing p and q. In particular, the inverse func- 
tion is defined by 


Ty) = y? mod n (AS.34) 


The decrypting exponent d is found using Equation (A5.32), which is equivalent to the 
statement (in ordinary integer arithmetic) that 


de = ġ(n)Q +1 (A5.35) 
for some integer Q. Note that ¢ (n) is itself related to p and q by Equation (A5.31). Since 
y = x°, we may use Equations (A5.32) and (A5.33) to write 
yt pa xd 
= xPmor (A5.36) 
= ((x9™)2)x 


We now make use of a celebrated theorem of Euler, which states that for any positive 
integers x andn with x < n, we have 


x?” = 1 modn (A5.37) 
Hence, the use of Equation (A5.37) in (A5.36) yields the desired decryption: 
y =x (A5.38) 


We thus see that finding the inverse function f7' is easy, given knowledge of the prime 
numbers p and q. 

The security of the RSA cryptoalgorithm rests on the premise that any method of 
inverting the function f, is equivalent to factoring n = pq. This equivalence raises the 
question: Is an attack by factoring 2 computationally feasible? It appears that the answer 
is no, provided that the prime numbers p and q are on the order of 100 decimal digits 
each and that there is no revolutionary breakthrough in factoring algorithms. 


DIGITAL SIGNATURES” 


For an electronic mail system to replace the use of ordinary paper mail for business trans- 
actions, it must be possible for a user of the system to “sign” an electronic message. The 
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use of a digital signature provides proof that the message did originate from the sender. 
To satisfy this requirement, the digital signature must have the following properties: 


*> The receiver of an electronic message is able to verify the sender’s signature. 
> The signature is not forgeable. 
> The sender of a signed electronic message is unable to disclaim it. 


To implement digital signatures using the RSA algorithm, we may proceed as follows. 
A user in possession of the private key d may sign a given message block m by forming 
the signature 


s= m? mod n (A5.39) 


It is difficult to compute s unless the private key d is known. Hence, a digital signature 
defined in accordance with Equation (A5.39) is difficult to forge. Moreover, the sender of 
message cannot deny having sent it, since no one else could have created the signature 
s. The receiver proceeds by using the public key e to compute 

s? = (m°) mod n” 
= m” mod 2 (A5.40) 


= m mod 7 


where, in the last line, use is made of Equation (A5.32). Hence, the receiver is able to 
validate the sender’s signature by establishing that the computation of s mod # produces 
the same result as the deciphered message m. Thus, the RSA algorithm satisfies all the 
three necessary properties of a digital signature. 


| A5.7 Summary and Discussion 


Cryptography is a “hot” research area. This statement should not come as a surprise. 
Considering the fact that we are in an information society, the importance of cryptography 
as a security mechanism will continue to grow. In this appendix, we have presented an 
introductory treatment of this highly important subject. 

We may classify cryptography into secret-key cryptography and public-key cryptog- 
raphy, depending on whether the key used for the encryption of a message and its decryp- 
tion is completely secret or partly public. Alternatively, we may classify a cryptographic 
system into a block cipher or stream cipher, depending on the method of implementation. 
A block cipher exhibits error propagation, which can prove highly valuable in 
authentication. 

Among the many cryptographic systems developed to date, the data encryption stan- 
dard (DES) and the Rivest-Shamir—Adleman (RSA) algorithms stand out as the most suc- 
cessful ones. Both of these cryptoalgorithms are block ciphers. They differ from each other 
in that the DES algorithm involves the use of a secret key whereas the RSA algorithm 
involves the use of a public key. In a secret-key system, the same key is shared both by the 
sender and the receiver. On the other hand, in a public-key system, the key is split into 
two parts: a public key located in the transmitter and a private (secret) key located in the 
receiver; in the latter system, it is computationally infeasible to recover the plaintext mes- 
sage from its encrypted version without knowledge of the private key. 

Although public-key cryptosystems such as RSA provide an effective method for key 
management, they are inefficient for the bulk encryption of data due to low bandwidths. 
In contrast, conventional cryptosystems such as DES provide better throughput, but they 
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require key management. This suggests the possible use of a hybrid approach exploiting 
the best elements of both cryptosystems as the basis for the practical design of a secure 
communication system. For example, the RSA algorithm may be used for authentication, 
and the DES algorithm for encryption. 


i NOTES AND REFERENCES 


1. 


10. 


11. 


12. 


For an introductory treatment of cryptography, see Chapter 15 of the book by Adámek 
(1991). For a comprehensive treatment of the many facets of cryptology, see the book 
edited by Simmons (1992); this book is an expanded edition of a Special Issue of the 
Proceedings of the IEEE (1988) on cryptology. The chapter contributions of the book by 
Simmons are written by leading authorities on the subject of cryptology. A nice treatment 
of cryptology is also presented in the book by van Tilborg (1988). 


. The era of scientific secret-key cryptography was ushered in with the publication of a 


landmark paper by Shannon (1949), which established the connection between cryptog- 
raphy and information theory. 


. The era of public-key cryptography was established with the publication of another land- 


mark paper by Diffie and Hellman (1976), which showed for the first time that it is possible 
to have secret communications without any transfer of a key between sender and receiver. 
It was the paper by Diffie and Hellman that sparked the explosion of research interest in 
cryptology, which has continued ever since. 


. The term enemy cryptanalyst is commonly used in cryptology to refer to a cryptogram 


interceptor (eavesdropper); its usage originates from military applications. 


. For a comprehensive treatment of stream ciphers, see Chapter 2 written by R. A. Rueppel 


in the book Contemporary Cryptology, edited by Simmons (1992). 


. For a highly readable account of the Shannon model of cryptography, see the opening 


chapter by J. L. Massey in the book edited by Simmons (1992). 


. The one-time pad derives its name from its use (shortly before, during, and after World 


War H) by’spies of several governments, who were given a pad of paper with a randomly 
chosen key and told to use it only for a single encryption. The one-time pad is also known 
as Vernam’s cipher, so named in recognition of its originator, G. S. Vernam. 


. For a derivation of Equation (A5.11), see the original paper by Shannon (1949). 
. The history of the DES algorithm is recounted by M. E. Smid and D. K. Branstad in Chapter 


1 of the book edited by Simmons (1992). For a description of the DES algorithm, see Diffie 
and Hellman (1979). See also the books by Meyer and Matyas (1982) and Torrieri (1992, 
Chapter 6). 


For a comprehensive treatment of public-key cryptography, see Chapter 4 by J. Nechvatal 
in the book edited by Simmons (1992). This book also includes a chapter contribution by 
W. Diffie that describes the several attempts to devise secure public-key cryptoalgorithms 
and the gradual evolution of a variety of protocols based on them. 


The RSA system is patented; it is named in recognition of its originators R. L. Rivest, 
A. Shamir, and L. Adleman. The original reference for this cryptosystem is Rivest, Shamir, 
and Adleman (1978). 


The idea of a digital signature was first discussed by Diffie and Hellman (1976). Its imple- 
mentation using the RSA algorithm is described by Rivest, Shamir, and Adleman (1978). 
For a detailed treatment of digital signatures, see Chapter 6 by C. J. Michell, F. Piper, and 
R. Wild in the book edited by Simmons (1992). 








TABLES 


The twelve tables compiled in this final appendix cover the following: 
> ASCII code 
> Fourier and Hilbert transforms 
P Bessel functions 
> Error function 
> Selected modem standards 
> Trigonometric identities, series expansions, and integrals 


> Useful constants and recommended unit prefixes 
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Į TABLE A6.1 ASCII code 








Bit Position 
7 0 0 0 0 1 1 1 1 
6 0 0 1 1 0 0 1 1 
4 3 2 1 5 0 1 0 0 1 0 1 
0 0 0 0 NUL DLE SP 0 @ P \ p 
0 0 0 1 SOH DC1 ! 1 A Q a q 
0 0 1 0 STX DC2 id 2 B R b r 
0 0 1 1 ETX DC3 # 3 G S c s 
0 1 0 0 EOT DC4 $ 4 D T d t 
0 1 0 1 ENQ NAK % 5 E U € u 
0 1 1 0 ACK SYN & 6 F v f v 
0 1 1 1 BEL ETB i 7 G w g w 
1 0 0 0 BS CAN ( 8 H X h x 
1 0 0 1 HT EM ) 9 I Y i y 
1 0 1 0 LF SUB S : J Z j z 
1 0 1 1 VT ESC + ; K [ k { 
1 1 0 0 FF FS d < L N 1 $ 
1 1 0 1 CR GS = = M ] m } 
1 1 1 0 so RS r > N A n ~ 
1 1 1 1 SI US / ? O — o DEL 
ACK Acknowledge ENQ Enquiry NUL Null or all zeros 
` BEL Bell or alarm EOT End of transmission RS Record separator 
BS Backspace ESC Escape SI Shift in 
CAN Cancel ETB End of transmission block so Shift out 
CR Carriage return ETX End of text SOH Start of heading 
DC1 Device control 1 FF Form feed SP Space 
DC2 Device control 2 FS File separator STX Start of text 
DC3 Device control 3 GS Group separator SUB Substitute 
DC4 Device control 4 HT Horizontal tab SYN Synchronous idle 
DEL Delete LF Line feed US Unit separator 
DLE Data link escape NAK Negative acknowledge VT Vertical tab 


EM End of medium 
(From Couch, 1990, with permission of Macmillan.) 
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i TABLE A6.2 Summary of properties of the Fourier transform 


Property 


Mathematical Description 











1. Linearity 
2. Time scaling 


3. Duality 

4. Time shifting 

5. Frequency shifting 
6 


. Area under g(t) 
7. Area under G(f) 
8. Differentiation in the time domain 
9, Integration in the time domain 
10. Conjugate functions 


11. Multiplication in the time domain 


12. Convolution in the time domain 


agi(t) + bg2(t) = aGilf) + bG2(f) 
where a and b are constants 


= tiff 
al e(£) 
where 4 is a constant 
If g(t) = G(f), 
then G(t) = g(-f) 
glt — to) = G(f) exp(—j2 af to) 
exp(j2mf.t)g(t) = G(f — fi) 


[ew a= co 


=|" chaf 
Ds 
aria 
$ 0 
[sin dra san + Pan 


If g(t) = Gf), 
then rie = G*(-f) 


elfen) = GuAIGAf — A) dà 


I smet- 9 drs sunean 


amre ener A SET AE 
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{ TABLE A6.3 Fourier-transform pairs 

















Time Function Fourier Transform 
rect 8 T sinc{fT) 
sinc(2 Wr) = ree 5) 
1 
exp(—at)u(t), a>0 et for 
2 
exp(—a(z|), a>0 rae 
expl- mt’) exp(—af?) 
-del li| <T 
T? T sinc” fT) 

0, l= T 
&(t) 1 
1 5(f) 
Slt — to) exp(—j2 rf to) 
exp(j27f.t) af — f) 
cos(2rf.t) HES — f) + f+ fN 
f 1 
sin(2 rft) z IF — f) — af + f) 

1 

sgn(t) jaf 

1 : 
> =j sgn(f) 

1 1 

u(t) 5 5(f) + Raf 

2 ‘ 1 < n 
ite ah PR) 


Notes: u(t) = unit step function 
6(t) = delta function, or unit impulse 
rect(t) = rectangular function of unit amplitude and unit 
duration centered on the origin 
sgn(f) = signum function 
sinc(t) = sinc function 
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{j TABLE A6.4 Hilbert transform pairs* 


Time Function 


m(t) cos(2 aft) 
m(t) sin(2 ft) 


Hilbert Transform 


m(t) sin(2af.t) 
—mi(t) cos(2 aft) 





cos{2 rft) sin(2mf;t) 
sin(2 aft) —cos(2 7f-t) 
sin t 1 — cost 
t 
bod 
1 2 
rect(t) pm log 4 
t+= 
1 
aie) = 
1 t 
1+¢ 1+? 
1 —78(t) 





"In the first two pairs, it is assumed that m(t) is band- 
limited to the interval -W < f < W, where W < f.. 
Notes: &(t): delta function 

rect(t): rectangular function of unit amplitude and 

unit duration centered on the origin 

log: natural logarithm 
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i TABLE A6.5 Table of Bessel functions? 


n\x 


SNINUNARWNHR OS 


‘oO 


10 
11 
12 
13 
14 


0.5 


0,9385 
0.2423 
0.0306 
0.0026 
0.0002 











Jn(*) 
1 2 3 4 6 8 10 12 
0.7652 0.2239 —0.2601 —0.3971 0.1506 0.1717 —0.2459 0.0477 
0.4401 0.5767 0.3391 —0.0660 —0.2767 0.2346 0.0435  —0.2234 
0.1149 0.3528 0.4861 0.3641 —0.2429 —0.1130 0.2546  —0.0849 
0.0196 0.1289 0.3091 0.4302 0.1148 —0.2911 0.0584 0.1951 
0.0025 0.0340 0.1320 0.2811 0.3576 —0.1054 —0.2196 0.1825 
0.0002 0.0070 0.0430 0.1321 0.3621 0.1858 —0.2341 —0.0735 
= 0.0012 0.0114 0.0491 0.2458 0.3376 —0.0145 ~0.2437 
0.0002 0.0025 0.0152 0.1296 0.3206 0.2167 —0.1703 
_ 0.0005 0.0040 0.0565 0.2235 0.3179 0.0451 
0.0001 0.0009 0.0212 0.1263 0.2919 0.2304 
a 0.0002 0.0070 0.0608 0.2075 0.3005 
a 0.0020 0.0256 0.1231 0.2704 
0.0005 0.0096 0.0634 0.1953 
0.0001 0.0033 0.0290 0.1201 
= 0.0010 0.0120 0.0650 











“For more extensive tables of Bessel functions, see Watson (1966, pp. 666-697), and Abramowitz and Stegun (1965, pp. 
358—406). 
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i TABLE A6.6 The error function? 


u 


0.00 
0.05 
0.10 
0.15 
0.20 
0.25 
0.30 
0.35 
0.40 
0.45 
0.50 
0.55 
0.60 
0.65 
0.70 
0.75 
0.80 
0.85 
0.90 
0.95 
1.00 
1.05 


erfiu) 


0.00000 
0.05637 
0.11246 
0.16800 
0.22270 
0.27633 
0.32863 
0.37938 
0.42839 
0.47548 
0.52050 
0.56332 
0.60386 
0.64203 
0.67780 
0.71116 
0.74210 
0.77067 
0.79691 
0.82089 
0.84270 
0.86244 








u erflu) 
1.10 0.88021 
1.15 0.89612 
1.20 0.91031 
1.25 0.92290 
1.30 0.93401 
1.35 0.94376 
1.40 0.95229 
1.45 0.95970 
1.50 0.96611 
1.55 0.97162 
1.60 0.97635 
1.65 0.98038 
1.70 0.98379 
1.75 0.98667 
1.80 0.98909 
1.85 0.99111 
1.90 0.99279 
1.95 0.99418 
2.00 0.99532 
2.50 0.99959 
3.00 0.99998 
3.30 0.999998 





The error function is tabulated extensively in several 
references; see for example, Abramowitz and Stegun 
(1965, pp. 297-316). 








| TABLE AG.7 Selection of ITU voiceband (telephone line) modem standards 


Bit rate, b/s 
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Symbol rate, bauds 














ITU 
Standard? Type of modulation 
(a) Symmetric modems: V.21 Binary FSK = 

V.22 bis QPSK 

V.26 QPSK 

V.27 8-PSK 

V.32 16-QAM 

V.34 1024-QAM 





Nested-constellation 
of four 960-QAM 
constellations 
Digital 

V.34 High Speed 


V.34 High Speed 







(b) Asymmetric modems: V.90: Downstream 
Upstream 














*The suffix “bis” designates the second version of a particular standard. 


i TABLE A6.8 Trigonometric identities 


exp(+j0) = cos 0 + j sin @ 
cos 0 = $[exp(j6) + exp(—74)] 


1 
sin 6 = 3 [exp(j@) — exp(—78)] 


sin? 0 + cos*@= 1 

cos? 0 — sin? 0 = cos(26) 

cos? 0 = 3[1 + cos(26)] 

sin? 9 = 4[1 — cos(26)] 

2 sin 8 cos 4 = sin(28) 

sin(a + B) = sin a cos B + cosa sin B 

cos(a + B) = cos acos £ F sina sin B 
tana + tan B 

1 ¥ tana tan 8 

sin a sin B = 4[cos(a — £) — cos(a + B)| 

cos a cos B = 4[cos(a — £) + cos(a + B)] 

sin a cos B = $[sin(a — £) + sin(a + B)] 


tan(a + 8) = 





300 
1,200 
2,400 
4,800 
9,600 

28,800 

33,600 





300 
600 
1,200 
2,400 
2,400 
3,429 
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i TABLE A6.9 Series expansions 


Taylor series 


£la 








"(a a 
f(x) = fla) + T te — a) + EO apa EE aa 
where 
a” f(x 
fa) = fie) 
MacLaurin series 
+O, PO) 2 4 £0) A 
fle) = f0) Maga tat 
where 
pao = Le) 
x=0 
Binomial series 
atapata t Bo es.., [ax] <1 
Exponential series 
=1+ pE 24 
exp x = tage tes: 
Logarithmic series 
log(1 + x) = x — $x? + fx? — 
Trigonometric series 
TAPAPA i a, i.s 
sinx = Tid to 
= 1 a2, la 
cosx = 1-5 +3” - 
1 2 
=xteetoxit 
tanx=x+5x° + 75x 
1 3 
Aga et Sgt pe yd 
sin” x =x Pa 40* + 
1 1 
-1 y = y — = xi tix... < 
tant x=x— gx tex k |x| <1 
sinc x = 1 —— (axl? + = (mx 
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| TABLE A6.10 Integrals 


Indefinite integrals 





x sin(ax) dx = 4 [sin(ax) — ax cos(ax)] 


i x cos(ax) dx = 3 [cos(ax) + ax sin(ax)] 
f x exp(ax) dx = 4 exp(ax)(ax — 1) 
1 


| x exp(ax*) dx = z exp(ax?) 
f exp(ax) sin(bx) dx = aoe exp(ax)[a sin(bx) — b cos(bx)] 


il exp(ax) cos(bx) dx = sty exp(ax)[a cos(bx) + b sin(bx)] 


f dx aA Tug {bx 
Aba ab a 


x? dx x a _,f bx 
Zro p PM VG 











Definite integrals 
; z ained gy =F epl- ab), a>0,b>0 
cos(ax) - a 
f= Bax? dx ap Pl ab), a>0,b>0 
” _cos(ax) _ 
f vao 2p dx = ri [sin(ab) — ab cos(ab)], a>0,b>0 
Í sinc x dx -Í E 
o 0 2 


F exp(—ax*) dx -E a>0 
[e exp(— -axtyde = 2 IF, a 
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f TABLE A6.11 Useful constants 


Physical Constants 
Boltzmann’s constant 

Planck’s constant 

Electron (fundamental) charge 
Speed of light in vacuum 
Standard (absolute) temperature 
Thermal voltage 


k = 1.38 X 107” joule/degree Kelvin 
h = 6.626 X 107% joule-second 

q = 1.602 x 10° coulomb 

c = 2.998 X 10° meters/second 

Ty = 273 degrees Kelvin 

Vy = 0.026 volt at room temperature 


Thermal energy kT at standard temperature kT, = 3.77 X 107*4 joule 
One hertz (hz) = 1 cycle/second; 1 cycle = 27 radians 


One watt (W) = 1 joule/second 
Mathematical Constants 
Base of natural logarithm 
Logarithm of e to base 2 
Logarithm of 2 to base e 
Logarithm of 2 to base 10 

Pi 


e = 2,7182818 
log: e = 1.442695 
log 2 = 0.693147 
logyo 2 = 0.30103 
m = 3.1415927 





i TABLE A6.12 Recommended unit prefixes 


` Multiples and Submuttiples 





1012 
10° 
10€ 
10° 
1073 
107° 
107? 
107! 


Prefixes Symbols 
tera T 
giga G 
mega M 
kilo K (k) 
milli m 
micro L 
nano n 
pico P 


menee eee 





GLOSSARY 


i Conventions and Notations 


1. 


2. 
3. 
4. 


10. 


11. 


12. 


13. 


14. 


The symbol | | means the absolute value, or magnitude, of the complex quantity 
contained within. 

The symbol arg( ) means the phase angle of the complex quantity contained within. 
The symbol Re[ ] means the “real part of,” and Im[ ] means the “imaginary part of.” 


Unless stated otherwise, the natural logarithm is denoted by log. Logarithms to bases 
2 and 10 are denoted by log, and logio, respectively. 


. The use of an asterisk as superscript denotes complex conjugate, e.g., x* is the com- 


plex conjugate of x. 


. The symbol = indicates a Fourier-transform pair, e.g., g(t) = G(f), where a low- 


ercase letter denotes the time function and a corresponding uppercase letter denotes 
the frequency function. 


. The symbol F| ] indicates the Fourier-transform operation, e.g., F[g(t)] = G(f), 


and the symbol F~*[ ] indicates the inverse Fourier-transform operation, e.g., 
FIG) = g(t). 
The symbol %* denotes convolution, e.g., 


o 


x(t) X Alt) = a x(7)h(t — 7) dr 


The symbol @ denotes modulo-2 addition, except in Chapter 10 where binary arith- 
metic is used and modulo-2 addition is denoted by an ordinary plus sign throughout 
that chapter. 

The use of subscript To indicates that the pertinent function g7,(t), say, is a periodic 
function of time ¢ with period To. 

The use of a hat over a function indicates one of two things: 


(a) the Hilbert transform of a function, e.g., the function (t) is the Hilbert transform 
of g(t), or 

(b) the estimate of an unknown parameter, e.g., the quantity @(x) is an estimate of 
the unknown parameter a, based on the observation vector x. 

The use of a tilde over a function indicates the complex envelope of a narrowband 

signal, e.g., the function 2(t) is the complex envelope of the narrowband signal g(t). 

The exception to this convention is in Section 10.8, where, in the description of turbo 

decoding, the tilde is used to signify extrinsic information and thereby distinguish it 

from log-likelihood ratio. 

The use of subscript + indicates the pre-envelope of a signal, e.g., the function 

g.(t) is the pre-envelope of the signal g(t). We may thus write g,(t) = g(t) + jê(t), 

where ĝ(t) is the Hilbert transform of g(t). The use of subscript — indicates that 

g(t) = g(t) — jĝ(t) = g+” (t). 

The use of subscripts J and Q indicates the in-phase and quadrature components of 

a narrowband signal, a narrowband random process, or the impulse response of a 

narrow-band filter, with respect to the carrier cos(27rf,t). 
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15. For a low-pass message signal, the highest frequency component or message band- 
width is denoted by W. The spectrum of this signal occupies the frequency interval 
-W = f < Wand is zero elsewhere. For a band-pass signal with carrier frequency 
f. the spectrum occupies the frequency intervals, f - W = f = fe + W and 
-f.— W=f <—-f, + W, and so 2W denotes the bandwidth of the signal. The 
(low-pass) complex envelope of this band-pass signal has a spectrum that occupies 
the frequency interval -W = f = W. 

For a lowpass filter, the bandwidth is denoted by B. A common definition of filter 
bandwidth is the frequency at which the magnitude response of the filter drops by 
3 dB below the zero-frequency value. For a band-pass filter of mid-band frequency 
f. the bandwidth is denoted by 2B, centered on f,. The complex low-pass equivalent 
of this band-pass filter has a bandwidth equal to B. 

The transmission bandwidth of a communication channel, required to transmit a 
modulated wave, is denoted by Br. 

16. Random variables or random vectors are uppercase (e.g., X or X), and their sample 
values are lowercase (e.g., x or X). 

17. A vertical bar in an expression means “given that,” e.g., fx(x | Hp) is the probability 
density function of the random variable X, given that hypothesis Ho is true. 

18. The symbol E[ | means the expected value of the random variable enclosed within; 
the E acts as an operator. 

19. The symbol var[ ] means the variance of the random variable enclosed within. 

20. The symbol cov[ ] means the covariance of the two random variables enclosed 
within. 

21. The average probability of symbol error is denoted by P.. 

In the case of binary signaling techniques, p1o denotes the conditional probability 
of error given that symbol 0 was transmitted, and po; denotes the conditional prob- 
ability of error given that symbol 1 was transmitted. The a priori probabilities of 
symbols 0 and 1 are denoted by po and pı, respectively. 

22. The symbol | ) denotes the time average of the sample function enclosed within. 

23. Boldface letter denotes a vector or matrix. The inverse of a square matrix R is denoted 

by R71. The transpose of a vector w is denoted by w”. The Hermitian transpose of 

a complex-valued vector x is denoted by x”; Hermitian transposition involves both 

transposition and complex conjugation. 

The length of a vector x is denoted by || x ||. The Euclidean distance between the 

vectors x; and x; is denoted by d; = || x; — x; ||. 

The inner product of two real-valued vectors x and y is denoted by xy; their outer 

product is denoted by xy’. If the vectors x and y are complex valued, their inner 

product is x"y, and their outer product is xy”. 

26. The vector product of two M-by-1 vectors œ and B is an M-by-1 vector defined by 


a, By 


Bo 


24 


25 


a' B= 


amPm 
where a, and 6, are the kth elements of œ and B, respectively. The L, norm of the 


vector product a + B is defined by 
M 


læ. Blh = z Bin 
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Í Functions 


1 1 
1. Rectangular function: rect(t) = k aS : <z 
0, |ż | > 
. A 1, t>0 
2. Unit step function: u(t) = { 0, peo 
I, ti>0 
3. Signum function: sgn(t) = 0, t=0 
-1, t<0 
4, (Dirac) delta function: 5(t) = 0, t#0 


ic 5(t) dt = 1 





or, equivalently, | g(t) 5(t — to) dt = g(to) 
5. Sinc function: sinc(x) = sin(rx) 
TX 
6. Sine integral: Si(u) = f mae dx 
o x 
7. Error function: erf(u) = = : exp(—z*) dz 
Complementary error function: erfc(u) = 1 — erf(u). 
vot ; n\ _ n! 
8. Binomial coefficient (:) = u- Hk 


9. Bessel function of the first kind 
of order n: 


10. Modified Bessel function of the 
first kind of zero order: 


11. Confluent hypergeometric Spe dN ax , aati) x? 
function a ET TEETE 


1 {” hter : 
Jx) = in f expl jx sin 0 — jn@) d0 


1 m 
I(x) = a J- exp(x cos 6) d@ 


I Abbreviations 


A: ampere 

AC: alternating current 

ADC: analog-to-digital converter 

ADM: adaptive delta modulation 

ADPCM: adaptive differential pulse-code modulation 
ADSL: asymmetric digital subscriber line 

AM: amplitude modulation 

ANSI: American National Standards Institute 

APB: adaptive prediction with backward estimation 
APF: adaptive prediction with forward estimation 


AQB: adaptive quantization with backward estimation 
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AQF: 


ASCI: 
ASK: 
ATM: 
AWGN: 
b/s: 
BER: 
BISDN: 
BPF: 
BSC: 
CAP: 
CCITT: 


CDMA: 
CELP: 
CO: 
codec: 
CPFSK: 
` CRC: 

CW: 
DAC: 


dBW: 
dBmW: 
DC: 
DEM: 
DES: 
DFT: 
DM: 
DMT: 
DPCM: 
DPSK: 
DSB-SC: 


DS/BPSK: 


DSL: 
exp: 
FDM: 
FDMA: 
FEXT: 
FFT: 
FH: 


adaptive quantization with forward estimation 
automatic-repeat request 

American National Standard Code for Information Interchange 
amplitude-shift keying 

asynchronous transfer mode 

additive white Gaussian noise 

bits /second 

bit error rate 

broadband ISDN 

band-pass filter 

binary symmetric channel 

carrierless amplitude/phase modulation 
Consultative Committee for International Telephone and Telegraph (Now 
renamed the ITU) 

code-division multiplexing 

code-division multiple access 

code excited linear predictive (model) 

central office 

coder/decoder , 

continuous-phase frequency-shift keying 

cyclic redundancy check 

continuous wave 

digital-to-analog converter 

decibel 

decibel referenced to 1 watt 

decibel reference to 1 milliwatt 

direct current 


. demodulator 


data encryption standard 

discrete Fourier transform 

delta modulation 

discrete multitone 

differential pulse-code modulation 
differential phase-shift keying 
double sideband-suppressed carrier 
direct sequence/binary phase-shift keying 
digital subscriber line 

exponential 

frequency-division multiplexing 
frequency-division multiple access 
far-end crosstalk 

fast Fourier transform 

frequency hop 


FH/MESK: 


FMEB: 
FSK: 
GMSK: 
GSM: 
HDTV: 
Hz: 
IDFT: 


modem: 
MPEG: 


ps: 
MSK: 
NCO: 
NEXT: 
nm: 
NRZ: 
NTSC: 
OC: 
OFDM: 
OOK: 
OSI: 
PAM: 
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frequency hop/M-ary frequency-shift keying 
frequency modulator with feedback 
frequency-shift keying 

Gaussian filtered MSK 

global system for mobile communication 
high definition television 

Hertz 

inverse discrete Fourier transform 
intermediate frequency 

input/output 

internet protocol 

intermediate standard-95 

integrated services digital network 
intersymbol interference 

International Organization for Standardization 
International Telecommunications Union 
joint photographic experts group 
local-area network 

linear delta modulation 
least-mean-square 

natural logarithm 

logarithm to base 2 

logarithm to base 10 

linear predictive coding (model) 
low-pass filter 

maximum 4 posteriori probability 
maximum likelihood 

minimum mean-square error 
modulator—demodulator 

motion photographic experts group 
millisecond 

microsecond 

minimum shift keying 
number-controlled oscillator 

near-end crosstalk 

nanometer 

nonreturn-to-zero 

National Television Systems Committee 
optical carrier 

orthogonal frequency-division multiplexing 
on-off keying 

open systems interconnection 
pulse-amplitude modulation 
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PCM: 
PDM: 
PG: 
PLL: 
PN: 
POTS: 
PPM: 
PSK: 
PSTN: 
PWM: 
QAM: 
Qos: 
QPSK: 


rms: 
RS: 
RS-232 
RSA: 
RSC: 
RZ: 


SDH: 
SDMA: 
SDR: 
SNR: 


SONET: 


STM: 
Tc: 
TCM: 


TDMA: 


VCO: 


VLSI: 


WDM: 


pulse-code modulation 
pulse-duration modulation 
processing gain 

phase-locked loop 

pseudo-noise 

plain old telephone service 
pulse-position modulation 
phase-shift keying 

public switched telephone network 
pulse-width modulation 
quadrature amplitude modulation 
quality of service 

quadriphase-shift keying 

radio frequency 

root-mean-square 

Reed-Solomon 

Recommended standard-232 (port) 
Rivest-Shamir-Adelman 

recursive systematic conyolutional (code) 
return-to-zero 

second 

synchronous digital hierarchy 
space-division multiple access 
signal-to-distortion ratio 
signal-to-noise ratio 

synchronous optical network 
short-time Fourier transform 
synchronous transfer mode 


< time compression 


trellis-coded modulation 
time-division multiplexing 
time-division multiple access 
television 

ultra high frequency 

volt 

voltage-controlled oscillator 
very high frequency 
very-large-scale integration 
watt 

wavelength division multiplexing 
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binary symmetric channel (BSC), 
258, 629, 667 
bipartite graphs, 685 


bipolar code, 281 
bipolar return-to-zero (BRZ) 
signaling, 207 
B-ISDN. See broadband integrated 
services digital network 
bit, 204, 569 
bit-by-bit interleaving procedure, 
214 
bit duration, 253 
bit energy-to-noise density ratio, 
384 
bit error rate (BER) 
assumptions of, 209 
for coherent binary FSK, 384 
of digital modulation schemes, 
417 
probability of, 23 
in signal regeneration, 208 
and.symbol error probability, 
335-336 
bit-rate reduction, 218 
bit stuffing, 215 
blanking pulse, 5 
block cipher, 744, 745, 746 
block code, 590, 632 
distinguishing feature of, 627 
rate of, 685 
blocks, 572 
Bose-Chaudhuri-Hocquenghem 
_(BCH) codes, 653-654 
bounds 
for prediction, 332 
use of, 332 
Brattain, Walter H., 28 
British Broadcasting Corporation 
(BBC), 27 
broadband integrated services 
digital network (B-ISDN), 
14-15 
cells in, 15 
quality of service, 14 
and the telephone network, 14 
broadband networks, 14-15 
broadcasting mode, 2-3 
broadcasting system, 128 
burstiness, 7 
byte, 7 


Cc 

cable-television systems, 17 

Campbell’s theorem, 60 

CAP. See carrierless amplitude/ 
phase 


capacity, 568 
capacity boundry, 602 
capture effect, 148-149 
carrier and timing synchronization 
systems, 449 
carrier component, 135 
carrier frequency, 92 
carrier-frequency tuning, 128 
carrierless amplitude/phase (CAP), 
369, 380 
carrierless amplitude/phase (CAP) 
modulation, 373, 431 
bandwidth for, 375 
idea behind, 369 
carrierless amplitude/phase (CAP) 
receiver 
digital implementation of, 
379-380 
improved performance of, 379 
in an unknown environment, 
379 
carrierless amplitude/phase (CAP) 
system 
application of, 380 
basic structure of, 378-379 
modulation, 369, 373, 375 
receiver, 379-380 
structure of, 378-379 
transmitter, 378 
carrierless amplitude/phase (CAP) 
transmitter, 378 
carrier phase, 403 
carrier phase recovery, 448, 458, 
459-463 
carrier synchronization, 448 
carrier-to-noise ratio, 137-139, 
144 
defined as, 138, 150 
large versus small, 141-142 
level of operation, 138 
low versus high, 149 
versus signal-to-noise ratio, 151 
carrier wave, 19-20 
Carson’s rule 
for approximate evaluation, 163 
and the universal curve, 119 
Cartesian product, 369-370 
cascade connection 
noise figure of, 526 
of two-port networks, 524-525 
catastrophic code, 667 
CDMA. See code division multiple 
access 


CDM systems, 505 
cell delay 
defined as, 14 
variation of, 14 
cell loss ratio, 14 
cells, 14-15 
cell splitting, 531 
cell-switching technology, 14 
cellular concept, 530 
cellular radio 
idealized model of, 530 
propagation problems of, 532 
wireless communications in the 
context of, 530 
CELP 
codec implementation, 553 
distinguishing feature of, 552 
encoder for, 552 
modeling of, 616 
See also code-excited LPC 
central limit theorem, 60, 498 
definition of, 56 
and the Gaussian process, 55 
central moments, 712 
channel, 2 
characteristics of, 309-310 
frequency response of, 261 
and random noise, 259 
channel bandwidth 
definition of, 3 
occupancy of, 347 
primary communication 
resource, 92 
used in North America, 102 
channel capacity, 587-589 
concept of, 630-631 
of a discrete memoryless channel, 
588 
channel capacity theorem, 599 
channel code word, 21 
channel coding 
design goal of, 589 
and mapping, 589 
techniques for, 628 
channel coding theorem, 589-593, 
616, 630-631 
application of, 591-592 
operations of, 628 
Shannon’s second theorem, 
590-591 
significance of, 592 
unsatisfactory feature of, 631 
channel data rate, 627 


channel decoder 
under designer’s control, 590 
goal of, 627 
inverse mapping operation, 590 
channel encoder, 626 
under designer’s control, 590 
goal of, 627 


introduction of redundancy, 590 


mapping operation, 590 
Markovian assumption for, 681 
channel imperfections, 2 
channel impulse response, 291° 
channel input, 597, 610 
channel matrix, 582 
channel model, 130 
channel noise, 31-32 
absence of, 228 
and bit error, 248 
condition of acting alone, 
282-283 
effects of, 209, 296 
in PCM systems, 209, 253 
reducing the effect of, 210 
source of, 32 
uncertainty due to, 403 
channel output, 72 
channel parameters, 587 
channel signal-to-noise ratio, 134 
for AM, 135 
definition of, 132 
formula for, 147 
characteristic function, 713 
Chebyshev inequality, 713 
check node, 685 
chip, 488, 501 
chip duration, 494 
chip rate, 501 
chrominance signal, 6 
cipher, 742 
ciphertext, 742 
circuit, 10-11 
circuit-switched network, 11 
controlled by, 11 
establishing a connection, 11 
circuit switching, 10-11 
circulant matrix, 442, 443 
circular constellation, 368 
Clark, Arthur C., 29 
clock recovery, 448 
closed-loop optimization 
procedure, 551 
coaxial cable 
application of, 17 
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consists of, 17 
versus twisted-pairs, 17 
co-channel cells 
determination of, 531 
finding of, 531 
as interference, 553 
code, catastrophic, 667 
code book, 580 
codec, 552 
coded binary pulses, 193 
code division multiple access 
(CDMA) 
advantage of, 514 
codes for, 505 
systems, 548 
code-division multiplexing (CDM) 
as an alternative method, 505 
bandwidth requirements of, 505 
defined as, 21 
coded pulse 
in analog modulation, 217 
in digital pulse modulation, 183 
use of, 184 
code elements, 203-204, 212 
code-excited LPC, 552 
code rate, 590, 627, 654 
code tree, 657, 657-658 
code vector, 660 
code word, 627 
average length, 574 
in binary form, 574 
duration of, 212 
code-word length, average, 574 
coding efficiency, 574 
coding gain, 425 
coding theory, 28, 676 
coherence bandwidth, 540 
coherent binary frequency-shift 
keying (FSK), 380 
coherent binary FSK system, 384 
characterized by, 381 
generation and detection of, 384 
receiver, 384-385 
coherent binary phase-shift keying 
(PSK) 
bit error rate, 352, 417 
characteristics of, 350 
coherent binary PSK system, 384 
bit error of, 357 
characteristics of, 350 
receiver, 352 
signals, 352 
coherent detection, 98, 131, 133 
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coherent detection (Continued) 
and demodulation, 95 
effect of, 97-98 
use of, 132 
coherent M-ary PSK, 367 
coherent M-ary quadrature 
amplitude modulation (QAM), 
464 
coherent MSK 
bit error rate for, 394 
expressions for, 417 
coherent phase-shift keying (PSK), 
349 
coherent QPSK 
bit error rate for, 417 
symbol error probability, 358 
coherent QPSK system 
signals of, 359 
specifications of, 458 
coherent quadriphase-shift keying 
(QPSK), 354 
colored noise channel, 607-611 
color receptors 
in the human eye, 6 
types of, 6 — 
communication, 1 
applications of, 1 
frequencies for, 4 
fundamentals of, 2-3 
types of, 21-23 
communication, error-free, 568 
communication channel, 7, 15-19, 
277 
classification of, 3, 15, 19 
description of, 15-19 
simultaneous use of, 512 
use of, 88 
communication link 
analysis of, 517 
in circuit switching, 11 
communication networks, 10-15 
communication process, 1-3 
communication resources, 3 
communications satellites 
in geostationary orbit, 19 
historical notes, 26-29 
role of, 19 
“second generation”, 515 
communication system 
common feature of, 3 
design of, 21 
elements of, 2 
noise analysis of, 3, 64, 523 
primary resources of, 3 


purpose of, 19, 88 
source of limitations, 248-252 
transition from analog to digital, 
183 
communication system designer, 23 
community-antenna television 
(CATV) system, 17 
commutator, 211 
compander, 203 
companding circuitry, 203 
companding law, fifteen-segment, 
426 
complementary error function, 255, 
256, 334 
complex envelope, 347, 727, 728, 
729, 730 
complex exponential Fourier series, 
717 
complex Fourier coefficient, 717 
complex least-mean-square (LMS) 
algorithm 
advantages of, 558 
limitations of, 558 
See also least-mean-square 
algorithm 
composite signal, 105, 149 
compound codes, 683-684 
compression algorithms, standard, 
8 
compression laws, 202-203 
compressor, 203 
computer communications, 2 
computer-generated data, 7 
conditional likelihood function, 
403-404 
conditional mean requirement, 200 
conditional probability, 706-707 
conditional probability density 
function, 320-321, 383, 
451-452, 710 
conditional probability of error, 
255, 256 
conditional probability of symbol 
error, 333 
confluent hypergeometric function, 
740-741 
confusion, 749 
conservation of time, 211-212 
constant angular velocity, 108 
constant envelope, 111 
constellation encoder, 444 
constrained optimization problem 
definition of, 437 
solving of, 609 


continuous AWGN channel, 
318-319 
continuous-phase frequency-shift 
keying (CPFSK), 381, 387 
continuous-phase frequency-shift 
keying (CPFSK) signal 
components of, 389 
deviation ratio of, 388 
phase of, 388 
representation of, 387 
continuous random variable, 594, 
708 
continuous source, 615 
continuous-time channel 
partitioning, 432-436 
continuous-wave (CW) modulation. 
20 
effects on reception, 130 
families of, 88-90 
principles of, 162 
techniques for, 130 
continuous-wave (CW) modulation 
system, 89-90 
comparison of, 132 
components of, 88-89 
noise in, 130 
performance of, 164 
control symbols 
in ASCI, 6 . 
for communication purposes, 6 
for printing of characters, 6 
conventional coherent binary FSK 
bit error rate expressions for, 
417 
with one-bit decoding, 417 
convolutional code, 654—656 
constraint length of, 655 
distance properties of, 663 
distinguishing feature of, 627 
maximum likelihood decoding 
of, 660-663 
performance of, 663 
use of, 654-656 
convolutional coding, 669 
convolutional encoder 
code tree for, 657-658 
input-output relation of, 660 
state of, 657-658 
trellis for, 658-659 
use of, 425 
convolution integral, 42-44, 718 
correlation coefficient, 342, 471 
correlation functions, 35-41 
correlation matrix, 40 


correlation receiver, 326-328, 329 
correlative-level coding, 266-271 

basis of, 268 

generalized form of, 274-275 

idea illustrated, 267 

premise of, 267 
correlator, 24-25 

inputs of, 549-550 

outputs of, 319-322 
coset leaders, 638-639 
Costas loop 

generalization of, 454 

for phase recovery, 454 
Costas receiver 

consists of, 96~97 

phase control in, 97 

use of, 96-97 
cost function, 558 
covariance function, 35-36 
CPFSK. See continuous-phase 

frequency-shift keying 

Cramér-Rao bound 

defined as, 462 

modification of, 462 
CRG code. See cyclic redundancy 

check (CRC) code 

critical band, 234. 
cross constellations, 369-370, 372 
cross-correlation functions, 40, 52 
cross-spectral densities, 52 
crosstalk, 21 

cause of, 279 

defined as, 501 

as impairment, 279 

types of, 279-280 
cryptanalysis, 742 

and authorized user, 742-743 

definition of, 746 

description of, 742 
cryptogram, 742-743 
cryptographic system 

classes of, 744 

classifications of, 759 

consists of, 743 

definition of, 743 

services of, 742 
cryptography, 617, 742 

and authentication problem, 743 

classifications of, 759 

data compression in, 749 

fundamental assumption, 745 

importance of, 759 

and secrecy problem, 743 
cryptology, 742 


crystal-controlled oscillator, 120 
cumulative distribution function, 
708 
cyclic code, 641-643 
advantage of, 641-642 
characteristics of, 652-654 
classes of, 652-654 
encoder for, 645-646 
generation of, 643 
properties of, 642 
in systematic form, 645, 646 
cyclic prefix, 441 
cyclic property, 642 
cyclic redundancy check (CRC) 
code 
for error detection, 652 
generator polynomials of, 653 


D 
damping factor, 160 
data-aided synchronization, 449 
data bits 
binary pattern of, 6 
for error detection, 7 
data communication, 7 
data compaction, 8, 575 
achieved by, 575 
assessing, 616 
schemes for, 575 
data compression, 7, 614-616 
in cryptography, 749 
forms of, 7-8 
idea of, 614 
as a lossy operation, 614-615 
reason for using, 615 
system components, 749 
techniques for, 218 
data compressor, 614 
data encryption, 617 
data encryption standard (DES), 
751-755, 759 
data encryption standard (DES) 
algorithm, 754 
data-modulated carrier, 500 
data multiplexers, 7 
data network, 11 
data signaling rate, 426 
data transmission system 
asynchronous versus 
synchronous, 7 
capabilities of, 446 
performance of, 293 
decision device, 259 
decision-directed mode, 290-291 
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decision-directed recursive 
algorithm, 450 
decision errors, 210 
decision feedback, 270 
decision feedback equalization, 
291-293, 379, 430 
decision feedback equalizer (DFE) 
consists of, 292 
error propagation in, 292-293 
feedback section of, 292 
feedforward section of, 292 
decision-making criterion, 323 
decision-making device 
designs of, 277 
operation of, 25 
decision rule, 350-351, 357 
applying, 382 
defined as, 661 
as the MAP rule, 323~324 
as the maximum likelihood rule, 
324 
used by the coherent detector, 
497 
decision threshold, 194 
decision tree, 575-576 
decoder, 446 
condition for optimality, 
200-201 
consists of, 552 
function of, 552 
units of, 235 
decoding algorithms, 678 
decoding complexity, 684 
decoding decisions, 663 
decoding error, 660 
decoding process 
methods of, 693 
and pulse generation, 208 
requirements of, 261 
decoding rule, 660 
decoding spheres 
maximum number of, 600 
packing of, 600 
decoding window, 663 
decommutator, 211 
decorrelation time, 37 
decryption, 742 
de Forest, Lee, 27 
delay, average, 540 
delay power spectrum, 539 
delay spread, 539 
as a channel impairment, 553 
defined as, 540 
effect of, 556 
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delta function, 62 
property of, 716-717 
sifting property of, 190, 320 
delta modulation (DM), 218-221 
advantage of, 219, 237 
and digital pulse modulation, 
237 
principles of, 218-219 
quantization error of, 220 
simplicity of, 223 
and transmitters, 228 
delta-sigma modulation, 221-223 
demodulation, 20, 88 
method of, 99 
stages of, 491 
demodulation scheme, 132 
demodulator output, 92 
demultiplexer 
in receiver, 359, 445-446 
in transmitter, 444 
demultiplexing system, 125-126 
DES. See data encryption 
standarddetection 
and error correction, 628 
of a pulse signal, 248 
detector, 326, 349 
deviation ratio, 387 
definition of, 119 
versus modulation index, 119 
DFT. See discrete Fourier transform 
diagonal matrix, 443 
dibits, 276 
difference-frequency term, 158 
differential detector 
components of, 364 
tangent type, 364-365 
differential encoder 
of the binary wave, 414 
consists of, 421-422 
method used, 207 
requirement of, 207 
differential entropy, 593-597 
differential phase encoder, 415 
differential phase modulation, 422 
differential phase-shift keying 
(DPSK), 407, 414-417 
bit error rate of, 417 
generation and detection of, 415 
receiver, 416 
transmitter, 415—416 
differential pulse-code modulation 
(DPCM), 227-229 
basic idea of, 227 
and digital pulse modulation, 
237 


system comparison, 228 
and transmitters, 228 
differential quantization scheme, 
227, 228 
differentiator, 143 
Diffie-Hellman public key- 
distribution system, 756 
diffraction, 17-18 
diffusion, 749 
digital audio broadcasting, 448 
digital circuit technology, 189 
digital communication 
basic form of, 309-310 
and bit error rate, 24 
and design goals, 354 
elements of, 24-25 
receiver, 337 
reliability of, 23, 24-26 
requirements of, 23 
and system design, 22 
task of designer, 626 
use of, 21 
digital data transmission, 247 
digital filter, second-order, 
454-455 
digital hierarchy, 214 
digital modem 
bidirectional, 428 
capabilities of, 428 
and data rates, 429 
design constraints of, 426 
fundamental design philosophy 
of, 426 
one realization of, 426~427 
signaling scheme for, 427 
solution to design problems, 428 
theoretical basis for the design 
of, 429 
digital modulation schemes 
comparison of, 417-420 
probability of error, 417 
types of, 346 
using a single carrier, 417-420 
virtues of, 347 
digital modulation techniques 
operation of, 448 
types of, 345-346 
digital multiplexers, 214-215 
design problems, 215 
major groups of, 214 
digital multiplexing-demultiplexing 
operation, 214 
digital passband transmission 
system, 344 
assessing performance of, 335 


performance degradation of, 544 
digital PSTN, 426 
digital pulse modulation 
basic form of, 193 
feature of, 236 
transmission of, 183 
digital satellite communication, 419 
digital signals, 214 
digital signal zero (DSO), 214 
digital signature 
for electronic mail systems, 
758-759 
properties of, 759 
use of, 758-759 
digital subscriber line (DSL) 
as a growing application, 277 
line codes for, 280-281 
operational environment of, 277, 
447 
and twisted pairs, 277, 297 
versus voiceband modems, 446 
digital switch, 215, 446 
digital-to-analog converter (DAC), 
445 
digital transmission facility, 215 
digital wireless communication 
systems, 550-551 
Dirac delta function. See delta 
function 
direct broadcast satellite (DBS) 
simplicity and affordability of, 
517 
direct broadcast satellites (DBS) 
use of, 517 
direct frequency modulation, 
120-121, 396 
directive gain, 520 
directivity, 520 
direct matrix inversion (DMI), 558, 
559 
direct-sequence M-ary phase shift 
keying (DS/MPSK), 508 
direct-sequence spread binary 
phase-shift-keyed (DS/BPSK) 
signal, 490, 492 
direct-sequence spread spectrum 
with coherent BPSK, 490-493 
principles of, 480 
systems, 498 
Dirichlet’s conditions, 715 
discrete cosine transform (DCT), 8 
discrete cosine transform 
coefficients, 8 
discrete Fourier transform (DFT), 
445 


defined as, 442 
and digital signal processing, 443 
discrete memoryless channel, 
581-584, 629-631 
channel capacity of, 588 
defined as, 581-582 
discrete memoryless source, 570 
extension of, 572 
properties of, 568 
discrete multitone (DMT), 431, 
440-443, 444-446 
applications of, 446 
basic idea of, 441 
and multichannel modulation, 
447-448 
use of, 441 
discrete pulse-amplitude 
modulation (PAM), 259 
discrete pulse modulation, 259 
discrete random variable, 708 
discrete source, 615 
discrete-time, memoryless Gaussian 
channel, 597 
discrete-time channel, 291 
discrete-time convolution, 627 
discrete-time Fourier transform, 
185 
discriminator, 144 
discriminator output, 145, 155 
dispersive channel, doubly, 542 
distance transfer function, 666 
distortion, 2 
acceptable, 614 
methods of reduction, 103 
produced by, 102-103 
unavoidable, 611-612 
distortion, amplitude, 191 
distortionless baseband binary 
transmission, 261-262 
distortion measure, 199 
distribution function 
properties of, 708 
of a stationary random process, 
34 
diversity techniques, 544-547 
performance with, 546-547 
specialized techniques, 559 
“divide and conquer”, 431 
DML. See direct matrix inversion 
DMT. See discrete multitone 
Donald Duck voice effect, 99-100 
Doppler shift, 535 
Doppler spectrum, 540-541 
Doppler spread, 539, 541 
double-frequency term, 158 


double sideband-suppressed carrier 
(DSB-SC) modulated signal 
(wave), 95, 96 
double sideband-suppressed carrier 
(DSB-SC) modulation, 133, 
134 
definition of, 93 
generated by, 94 
transmission of sidebands, 163 
double sideband-suppressed carrier 
(DSB-SC) receiver 
compared to an AM receiver, 
136-137 
model of, 132-133 
doubly dispersive channel, 542 
down conversion, 105 
downconverter, 448 
downlink, 19, 514-515 
downstream data transmission, 
281-282 
DPSK. See differential phase-shift 
keying 
DS/BPSK waveform, 490, 492 
DSL environment, 447 
DS/MPSK system, 508 
dual code, 641 
duobinary code, modified, 281 
duobinary coding, 270 
duobinary conversion filter, 
268-269 
duobinary encoder, 267-268 
duobinary signaling scheme, 
267-271 
frequency response of, 268 
technique, 271-272 
duobinary technique, 274-275 
“dynamic” multipath environment, 
532-533 


E 
echo cancellation, 277-278 
comparison of schemes, 278-279 
mode of operation for, 277-278 
echo canceller 
in transceiver, 278-279 
use of, 516 
effective aperture, 521 
effective radiated power referenced 
to an isotropic source (EIRP), 
521 
Einstein-Wiener-Khintchine 
relations, 46 
EIRP. See effective radiated power 
referenced to an isotropic 
sourceelastic store, 215 
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electromagnetic interference (EMI), 
17 
electron beam, 4 
elementary event, 704 
encoded text, 6 
encoder 
condition for optimality of, 
199-200 
functional units of, 235 
main parts of, 551 
operation of, 646 
states of, 659, 667 
encoding process 
operations of, 9 
steps of, 551 
use of, 203~204 
encryption, 742 
enemy cryptanalyst 
forms of attack by, 745 
intrusion of, 743, 744 
energy gap, 99 
energy signals, 312 
energy spectral density, 48, 353 
ENIAC, 28 
ensemble average 
autocorrelation function, 284 
estimation of, 41 
parameter, 75 
substituting time averages for, 41 
entropic coding redundancy, 9 
entropy 
conditional, 584 
definition of, 569 
formula for, 568 
properties of, 570-571 
entropy, conditional, 584 
envelope 
defined as, 730 
and phase components, 67-69 
types of, 730 
envelope delay, 16 
envelope detection, 102-103, 131 
envelope detector, 123, 143 
consists of, 92 
found in, 92 
loss of message in, 138 
need for, 406 
performance of, 137 
signal comparison, 141-142 
envelope distortion, 90-91 
equalizer, 191 
equiprobable symbols, 571 
equivalent noise temperature, 61, 
524-525 
ergodic process, 41—42, 51 
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error 
minimization of, 551 
possible kinds, 254 
probability of, 497-499 
error burst, 652 
error control, 626 
error-control code 
theory of, 485 
types of, 627 
error-control coding 
classes of, 626 
for reliable communication, 567 
techniques, 626 
techniques for, 683, 693 
use of, 626, 627 
error-detection bit, 7 
error-free communication, 568 
error function, complementary, 255 
error minimization, 551 
error pattern, 635 
error probabilities, conditional, 352 
error propagation 
elimination of possibility, 273 
phenomenon, 270 
property, 745 
error rate, 253 ` 
error signal 
calculation of, 457 
definition of, 288, 453 
for timing recovery, 457 
use of, 557 
error-syndrome vector, 635-636 
error threshold, 209-210 
error vector, 635 
estimation procedure, 517 
Euler’s formula, 453 
excess bandwidth factor, 441 
excess mean-square error, 291 
excitation generator, 551, 552 
excitation time, 718 
expander, 203 
expansion laws, 203 
exponential law, 193 
extended code 
average code-word length, 578 
use of, 577 
extended prefix code, 578 
extended source, 572 
extended-threshold demodulators, 
152, 153 
extraction, 261 
extrinsic information, 679 
eye opening, 293 
eye pattern 


definition of, 293 

as an experimental tool, 293 

interpretation of, 293 

and performance information, 
293 


F 
facsimile (fax) machine 
basic principle of, 6 
purpose of, 6 
in a receiving mode of operation, 
420 
fade rate, 541 
fading channel 
characteristics of, 541 
effects of, 545 
fading multipath channel, 536-539 
far-end crosstalk (FEXT), 279-280 
Farnsworth, Philo T., 27 
fast Fourier transform (FFT) 
algorithm, 443-444 
fast-frequency hopping, 502-503, 
504 
FDM. See frequency division 
- multiplexing 
FDMA. See frequency division 
multiple access 
FDMA system, 516 
FDM system 
block diagram of, 105-106 
modulation steps in, 107 
FEC. See feed-forward error 
correction 
feedback shift register, 480, 481 
feedback system, second-order, 160 
feed-forward error correction 
(FEC), 628, 629 
Fessenden, Reginald, 27 
FH/MFSK system 
fast versus slow, 502-503 
jamming effect on receiver, 502 
symbol error in, 502 
field-power pattern, 521 
figure of inerit, 134 
for amplitude modulation, 136 
definition of, 132, 193 
for frequency modulation, 147 
filtering, 128, 208-209 
filtering scheme, 100 
fine synchronization, 493 
finite-duration impulse response 
(FIR) filter, 379 
finite-state machine, 654 
fixed channel input, 582 


fixed channel output, 582 
fixed modulation scheme, 627 
fixed point-to-point links, 18-19 
fixed scatterers, 536 
flat-fading channel, 71-72 
flat-flat channel, 542 
flat Rayleigh fading channel, 554 
flat-top samples, 191 
Fleming, John Ambrose, 27 
flip-flops, 646 
flyback. See horizontal retrace 
FM demodulator 
with negative feedback, 153 
and oscillator types, 152 
FMFB demodulator, 152, 153, 154 
FMFB receiver, 154 
FM receiver 
breaking point of, 149 
interference suppression in, 
148-149 
model of, 142-143 
noise analysis of, 146 
noise in, 142 
threshold effects in, 152 
FM signal 
average power of, 115 
complex envelope of, 114 
demodulation of, 121-124 
desirable properties, 397 
detection of, 397 
distinguishing from AM signal, 
109 
effective bandwidth for, 117-119 
fundamental characteristic of, 
110 
generation of, 120-121 
side frequencies of, 117 
spectral analysis of, 110 
spectrum of, 115 
in theory and practice, 117 
FM signal, single-tone, 112-113 
FM stereo 
multiplexing, 124-126 
specification of standards, 124 
transmission, 124 
FM system 
emphasis in, 154-156 
nonlinear effects in, 126-128 
See also frequency modulation 
(FM) system 
FM threshold effect, 149-152 
FM threshold reduction, 152-154 
FM wave 
bandwidth requirement, 118 


with reduced modulation index, 
152-153 
forward error-control coding, 628, 
668 
forward error correction (FEC), 
626 
forward estimation, 681 
forward link, 547 
Fourier analysis, 715-720 
Fourier series expansion, 317 
Fourier series representation, 114 
Fourier transform 
definition of, 715 
inverse, 715 
of periodic signals, 717-718 
properties of, 716 
theory of, 716 
fractionally spaced equalizer (FSE), 
287 
frame, 552 
make-up of, 5 
method of synchronization, 
215-216 
structure, 547 
frame packing, 9 
frame-packing unit, 236 
free distance, 663 
free propagation 
channels based on, 15 
types of, 15 
free-space loss, 522 
free-space propagation model, 
518-523 
frequency demodulation 
defined as, 121 
methods of, 121 
frequency deviation, 110, 152 
frequency-discrimination method 
stages of, 98-99 
use of, 100 
frequency discriminator, 121, 
124 
consists of, 121-122 
input, 149 
requirements of, 99 
frequency diversity, 544-545 
frequency division duplexing 
(FDD), 547 
frequency division multiple access 
(FDMA), 513, 516 
frequency-division multiplexing 
(FDM) 
defined as, 20-21, 105 
method of modulation in, 106 


frequency-domain description, 444, 
720 
frequency down converter, 105, 
516 
frequency flat, 541 
frequency-flat channel, 542 
frequency-hop M-ary frequency 
shift-keying (FH/MFSK), 508 
frequency hopping, 500 
frequency-hop spread spectrum, 
499, 500 3 
communication systems, 500 
principles of, 480 
frequency-modulated wave, 413 
frequency modulation (FM), 20; 
27, 729 
capability of, 165 
cases of, 111 
characteristic of, 149 
definition of, 108-109 
direct, 120-121 
and mixing, 500 
as a nonlinear process, 109 
theory of, 126 
frequency modulation (FM) system 
noise analysis of, 142-147 
similarities to PPM system, 193 
frequency multiplication ratio, 
121 
frequency multiplier 
consists of, 120 
diagram of, 120-121 
frequency parameters, 128 
frequency response 
choice of, 155 
to denote, 44-45 
frequency reuse, 530 
frequency-shift keying (FSK) 
basic signaling scheme, 344-345 
and design of modems, 421 
and frequency modulation, 345 
represented by, 464 
frequency-shift keying (FSK) 
schemes, 418 
frequency-shift keying (FSK) signal, 
386 
frequency translation, 103, 
103-105 
frequency up converter, 105 
Friis formula, 526 
Friis free-space equation 
defined as, 522 
used for, 522 
FSK. See frequency-shift keying 
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full amplitude modulation, 
162-163 
full-cosine rolloff characteristic, 
266 
full-duplex link, 628 
functional 
defined as, 54 
versus function, 54 
fundamental frequency, 717 
fundamental inequality, 571 


G 
gain, 720 
gap, 432 
Gaussian assumption, 498 
Gaussian channel, 597 
Gaussian-distributed random 
variable, 54 
Gaussian distribution, 54, 72-73 
Gaussian filter, 397 
Gaussian-filtered minimum shift 
keying (GMSK) 
as a special kind of binary 
frequency modulation, 398 
undesirable feature of, 398 
Gaussian-filtered minimum shift 
keying (GMSK) modulator 
frequency shaping pulse of, 397 
and intersymbol interference, 
398 
Gaussian-filtered minimum shift 
keying (GMSK) signal, 397 
power spectrum of, 400 
spectral compactness of, 
398—400 
Gaussian-filtered MSK, 396—400 
Gaussian function, 397 
Gaussianity, 75 
Gaussian model, 55 
Gaussian process, 54-58 
definition of, 57 
mathematical justification, 55-56 
in the study of communications, 
55 
useful properties of, 56-58 
virtues of, 55 
Gaussian random variable, 54, 58 
generator equation, 634 
generator polynomial, 645 
of a cyclic code, 643 
definition of, 656 
geometric mean, 436 
geometric representation of signals, 
311 
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geometric signal-to-noise ratio, 436 
geostationary satellite 
communications system, 
514-515 
global coverage, 512 
Global System for Mobile 
Communications (GSM), 548 
frame efficiency of, 548 
wireless communication system, 
548 
glottis, 4 
GMSK. See Gaussian-filtered 
minimum shift keying 
Gold’s theorem, 505 
Gold sequences (codes) 
class of, 505 
correlation properties of, 507 
good codes, 631 
Gram-Schmidt orthogonalization 
procedure, 315-317 
granular noise, 220 
and distortion, 221 
versus quantization noise, 221 
Gray coding scheme, 422-423 
Gray encoder, 276 
Gray-encoded dibits, 363 
GSM. See Global System for 
Mobile Communications 
guard bands, 513 
guard interval, 441 
guard time, 547 
guided propagation 
channels based on, 15 
types of, 15 


H ; 

half-cycle cosine pulse, 389 

half-cycle sine pulse, 389-390 

half-duplex link, 628 _ 

Hamming distance, 637, 661, 
666 

Hamming single-error correcting 
code, 653 

Hamming weight, 637, 666 

handover, 530 

hard-decision coding, 630 

hard-decision decoders, 629-630 

hard-decision demodulation, 669 

hard decisions, 630 

harmonic distortion, 112 

harmonic structure, 4 . 

head end, 17 

hearing mechanism, 4 

Hermitian transposition, 443 

Hertz, Heinrich, 26-27 


heterodyning function, 128 
hexagonal cellular geometry, 531 
high-performing CAP system, 375 
Hilbert transform, 374, 408, 
723-725 
properties of, 725 
of a signal, 724 
Hilbert-transform pair, 376, 724 
Hockham, G. A., 29 
hop rate, 501 
horizontal retrace, 5 
host, 13 
Huffman code 
algorithm used to synthesize, 
578 
as a class of prefix codes, 578 
drawback of, 580 
nonuniqueness of, 579 
Huffman coding, 578-580, 616 
basic idea of, 578 
compared to Lempel-Ziv 
algorithm, 581 
and data compression, 8 
as entropic coding, 8 
Huffman decoding, 8-9 
Huffman encoding process, 578, 
579 
human auditory system, 234 
human communication, 4 
hybrid-modulated signal, 123 
hybrid modulation process, 374 
hybrid transformer 
definition of, 278 
simplified circuit of, 278-279 


I 
I-channel, 97 
ideal baseband pulse transmission, 
262 
ideal delay element, 268 
ideal envelope detector, 135 
ideal frequency discriminator, 124 
ideal narrowband filter, 45 
ideal Nyquist channel, 262-264, 
265-266 
difficulties of, 263-264 
use of, 263-264 
ideal sampled signal, 184 
ideal slope circuit 
characterized by, 121-122 
frequency response of, 122 
ideal system, 601 
identity matrix, 329 
image interference, 129 
impossible event, 704 


impulse function, 62 
impulse noise, 446 
impulse response, 656, 718 
index of performance, 224 
indirect frequency modulation, 
120-121 
individual demodulators, 106 
infinite bandwidth, 602 
information, 2 
information-bearing signal, 31-32, 
88 
in the digital domain, 277 
multiplying by the PN signal, 
488 
information capacity, 598, 616 
of a channel, 597-598 
defined as, 23, 598 
evaluation of, 598 
increasing of, 599 
information capacity theorem, 616 
application of, 607 
argument for, 599-600 
as a colored noise channel, 607 
and Gaussian channels, 597 
implications of, 601-603 
system parameters of, 599 
water-filling interpretation of, 
610 
information-theoretic concepts, 
572 
information theory 
fundamental limits in, 567 
important result of, 591 
as a mathematical discipline, 
567 
Shannon’s landmark paper, 567 
information transmission, 581 
information vector, 633 
inner conductor, 17 
inner product, 313, 314 
innovation symbol, 581 
in-phase channel, 408 
in-phase coherent detector, 97 
in-phase component, 93 
power spectral density of, 
395-396 
properties of, 65-66 
representation of, 67-69 
in-phase noise component, 131 
input alphabet, 582 
input signal-to-noise ratio, 134 
definition of, 131 
equation for, 497 
insertion loss, 16 
instantaneous codes, 577 


instantaneous frequency, 110 
definition of, 163 
equation for, 108 
instantaneous sampling, 184 
integration 
beneficial effects of, 221-222 
as a linear operation, 223 
interface, 11 
interference 
average power of, 495 
effect of, 490 
and fading, 71-72 
strength of, 148-149 
as unintentional or intentional, 
479 
interference suppression, 148-149 
interframe redundancy, 9 
interlaced fields, 5 
interlaced raster scan, 5 
interleaver 
definition of, 674 
types of, 674 
use of, 675 
intermediate frequency (IF), 128 
intermediate frequency (IF) band, 
18 
Internet, 13-14 
architecture of, 13-14 
evolution of, 28-29 
growth of, 29 
protocols for, 13-14 
Internet architecture 
functional blocks of, 13-14 
Internet protocol (IP), 13-14 
Internet Service Provider (ISP), 420 
and communication between 
PSTN, 425 
and public switched telephone 
network (PSTN), 420-425 
and voice modems, 420-422 
interpixel redundancy, 9 
interpolation formula, 186 
interpolation function, 186, 427 
intersymbol interference (ISI), 
259-261, 398 
and bit errors, 247 
as channel impairment, 379 
condition of, 282-283 
under designer’s control, 268 
as a dominant impairment, 279 
effects of, 294, 296 
as a form of interference, 296 
minimizing effects of, 260 
and noise presence, 294 
overcoming effects of, 441 


in peak distortion, 288 
and timing error, 266 
as an undesirable effect, 267 
intrinsic information, 679 
invariance, 331 
inverse discrete Fourier transform 
(IDFT), 442, 445 
inverse Fourier transform, 186, 
715 
inverse mapping, 589 
inverse-square law, 519 
irreducible polynomial, 505 
irregular codes, 691 
irregular interleavers, 691-692 
irregular LDPC code, 692 
irregular turbo code, 691, 692 


jammer, 493 

strategy of, 495 

types of, 508 

waveforms of, 508 
jammer, barrage noise, 508 
jammer, multitone, 508 
jammer, pulse noise, 508 
jammer, single-tone, 508 
jamming margin, 499 
jamming signal, 488 
jamming waveforms, 488 
jitter, 208 
joint distribution function, 33, 709 
joint moments, 713-714 
Joint Photographic Experts Group 

(JPEG), 8 
joint probability, 706, 707 
joint probability density function, 
594, 709-710 

joint probability distribution, 583 
JPEG image coding standard, 8 


K 

Kao, K. C., 29 

keys, 756 

key-schedule calculation, 753, 754 

keystream, 744, 745 

Kotel’nikov, V. A., 27 

Kraft-McMillan inequality, 
576-577 

Kummer’s differential equation, 


740 


L 

Lagrange multipliers 
method of, 437 
use of, 609 
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laser, 29 
layer, 11 
layered architecture, 11 
layer-to-layer interface, 13 
least-mean-square (LMS) 
algorithm, 288-290, 557 
for adaptive equalization, 
288-289 
and combined use, 297 
equations for, 289 
for linear adaptive prediction, 
226 
popularity of, 226, 227 
similarities, 290 
simplification of, 289 
summary of, 289 
uses of, 292 
using matrix notation, 289 
Leibniz’s rule, 257 
Lempel-Ziy algorithm, 8, 616 
compared to Huffman coding, 
581 
definition of, 580 
encoding process performed by, 
580 
standard for file compression, 
581 
Lempel-Ziv coding, 580-581 
light, 6 
likelihood functions, 322 
linear adaptive prediction, 225-227 
linear array signal processor 
to design, 554 
for the receiver, 554 
requirements of, 554 
linear block code 
basic property of, 634 
classes of, 653-654 
decoding procedure for, 639 
definition of, 632 
mathematical structure of, 
632-633 
minimum distance of, 637 
standard array of, 638 
linear combiner, 547 
linear delta modulator, 221, 
232-233 
linear diversity combining 
structure, 545 
linear equalization, 379, 556 
linear function, 54 
linearity property, 642 
linear modulation 
definition of, 93 
examples of, 163 
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linear modulation (Continued) 
forms of, 93-94 
types of, 93 
linear modulation systems, 729 
linear prediction, 223-227 
linear predictive coding (LPC), 551 
linear pre-emphasis and de- 
emphasis filters 
applications in, 157 
use of, 157 
linear receiver, 248 
design of, 283 
performance of, 132 
using coherent detection, 132 
linear system, 718 
linear time-invariant filter 
defined as, 250~251 
impulse response of, 44 
as a matched filter, 250-251 
use of, 248 
linear time-invariant system, 
719-720 
line codes 
candidates for, 281 
comparison of, 281 
for electrical representation, 
204-207 
power spectra of, 206 
selection of, 280-281 
types of, 205-207 
use of, 204—207 
line-scanning frequency, 5-6 
link budget, 518 
link budget analysis, 517 
link budget balance sheet, 517 
Lloyd-Max quantizer, 198, 
200-201 
loading problem, 438 
loading process, 438 
local loop, 420 
local oscillator, 97 
Lodge, Oliver, 27 
logarithmic function, 322 
log-likelihood function, 452 
defined as, 322 
defined for AWGN channel, 325 
relationship of, 322 
loop filter, 157, 159-160 
loop-gain parameter, 159 
lossless compression 
definition of, 7~8 
for digital text, 8 
versus lossy compression, 8 
lossless data compression, 575 


lossy compression 
definition of, 8 
the preferred approach, 8 
low-density parity-check (LDPC) 
codes, 683-686 
advantages of, 684 
block length of, 689 
construction of, 684-685 
decoding algorithm, 690 
decoding of, 689-690 
initialization of, 690 
minimum distance of, 689 
shared properties of, 693 
statistical analysis of, 689 
steps of, 690-691 
use of, 684 
low-noise amplifier, 515 
low-weight code words, 684 
LPC. See linear predictive coding 
Lucky, Robert, 28 
luminance signal, 6 


M 
magnitude response,,45, 608, 719 
magnitude spectrum, 715 
main lobe, 368, 720 
Manchester code, 207, 281 
many-to-one mapping, 8 
MAP decoder, 678 
Marconi, Guglielmo, 27 
marginal densities, 710 
marginal probability distribution, 
583 
Markov process, 678 
M-ary digital modulation 
techniques, 419-420 
M-ary frequency-shift keying 
(MFSK), 398—400 
consists of, 401 
for frequency hopping systems, 
500 
property of, 400 
M-ary FSK signal 
bandwidth efficiency of, 
401-402 
bandwidth requirements, 401 
orthogonal signals of, 401-402 
power spectra of, 401 
spectral analysis of, 401, 402 
M-ary PAM system 
in a channel bandwidth, 276 
consideration of, 276 f 
design complexity of, 277 
power requirements of, 276-277 


M-ary PSK 
comparison of, 419 
likelihood function for, 452 
power-bandwidth requirements 
for, 419-420 
signal constellations of, 365, 
420 
similar spectral and bandwidth 
characteristics, 419 
special case, 365 
symbol duration of, 367 
symbol error equation for, 
365-366 
M-ary PSK signal 
bandwidth efficiency of, 368 
baseband power spectral density 
of, 367 
power spectra of, 367 
as spectrally efficient, 402 
M-ary PSK systems, 449-450 
M-ary QAM 
detection for, 371 
functions in, 369 
performance of, 420 
symbol error probability for, 371 
transmitted energy in, 371 
M-ary QAM signal, 372 
M-ary quadrature amplitude 
modulation (QAM), 369-373 
M-ary signal, 402 
M-ary signaling scheme, 345-346 
M-ary system, 276 
masking threshold, 9, 234, 
234-235 
matched filter, 248-252, 286 
in the frequency domain, 251 
output, 406 
properties of, 251-252 
matched filter receiver 
correlation and, 326-327 
detector part of, 328 
mathematical models 
classes of, 31 
in probabilistic terms, 31 
matrixer 
difference signal generation, 125 
sum signal generation, 125 
maximal-length sequence 
autocorrelation function of, 482 
balance property of, 482 
choosing a, 484 . 
defined as, 482 
properties of, 482-484 
maximal-ratio combiner, 547 


maximal ratio combining principle, 
550 
maximum 4 posteriori probability 
(MAP) detection, 678 
maximum a posteriori probability 
(MAP) rule, 323-324 
maximum likelihood decision rule 
for an AWGN channel, 325 
purpose of, 325 
maximum likelihood decoder, 
322-326 
for computation, 324 
defined as, 661 
as an implementation device, 324 
theory of, 660-661 
maximum likelihood decoding rule, 
661 
maximum likelihood detection, 
330, 337 
maximum likelihood detectors, 435 
maximum likelihood estimation 
of the carrier phase, 453-458 
for problem solving, 449 
maximum likelihood rule, 324 
maximum likelihood signal 
detection, 346 
maximum-power transfer theorem 
applying, 61 
use of, 61 
Maxwell, James Clerk, 26 
mean, 35. 
mean Doppler shift, 541 
mean functions, 35-39 
mean output noise power 
definition of, 139 
equation for, 141 
mean output signal, 139 
mean-square distortion, 199 
mean-square error, 285 
as the cost function, 558 
definition of, 284 
mean-square error criterion 
for receiver design, 283 
uses of, 288 
median signal strength, 530 
melodic structure, 4 
memoryless channel, 321 
memoryless Gaussian channel, 
discrete-time, 597 
memoryless quantizer, 194 
Mersenne prime length sequences, 
485 
message, 155 
message bandwidth, 90 


message point, 322-323 
message polynomial, 643 
message signal, 132-133 
description of, 2 
generation of, 2 
message source, 348 
message spectrum 
for negative frequencies, 103 
requirement of, 99 
message vector, 660 
method of steepest descent, 
225-226 
microcells, 531 
microphone, 15-16 
Middleton, D., 27 
minimum average energy, 332 
minimum distance 
considerations of, 637—638 
definition of, 637 
minimum distance decoder, 661 
minimum energy signals, 331-332 
minimum energy translate, 332 
minimum mean square error 
(MMSE), 558 
criterion for, 558 
equilizer, 285 
receiver, 286-287 
minimum shift keying (MSK), 387 
as a form of binary FSK, 361 
signal-space diagram of, 
389-392 
minimum shift keying (MSK) signal 
power spectra of, 360 
mixer 
consists of, 103-104 
function of, 129 
operation of, 105 
mobile radio, 18, 529 
mobile radio channel 
capability of, 18 
as a linear time-varying channel, 
18 
propagation effects of, 18 
mobile switching center, 530 
mobility, 18 
modem, 7 
configuration of, 421 
as a conversion device, 420 
design of, 421 
and the Internet, 420 
portions of, 420 
modem, facsimile, 420 
modem configuration, 421 
modified Bessel equation, 738 
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modified Bessel function, 737-739 
modified duobinary code, 281 
modified duobinary coder 
responses of, 272, 273 
useful feature of, 272-273 
modified duobinary conversion 
filter, 273 
modulated signal (wave), 95 
modulating signal (wave), 88 
modulation, 19 
operations of, 628 
stages of, 490 
modulation format, 101-102 
modulation index 
defined as, 110 
restriction of, 112 
small values of, 119 
values of, 117-118 
modulation process, 19-21 
classification of, 20 
definition of, 88 
modulation scheme, bandwidth- 
conserving, 354 
modulation system, binary-coded, 
197 
modulator-demodulator, 420 
modulo-2n correction logic, 364 
moments, 712 
Morse, Samuel, 26 
Morse code, 26, 574 
Motion Picture Experts Group 
(MPEG), 9, 234 
moving-coil receiver, 15-16 
MPEG-1 audio coding standard, 9 
capabilities of, 234 
operation of, 235 
performance of, 234 
suitable for, 10 
MPEG-1 video coding standard, 9 
MPEG audio coding standard, 234, 
237 
MSK receiver, 394, 395 
MSK signal, 391 
characteristics of, 396 
demodulation of, 394 
detection of, 394 
error probability of, 392 
generation of, 394, 396 
possible forms of, 390 
power spectra of, 394-396, 398 
properties of, 396 
MSK system, 391 
MSK transmitter, 394, 395 
p-law, 202-203 
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multibeam antennas, 514 
multichannel data transmission 
system, 433, 434 
multichannel modulation, 440-441 
basic idea of, 431 
form of, 431, 465 
multichannel transmission system, 
437-438 
multilevel encoding, 348, 378 
multiloop feedback circuit, 480 
multipath 
physical phenomenon of, 513 
presence of, 513 
multipath autocorrelation profile, 
537 
multipath channel 
as a channel impairment, 553 
classification of, 541-542 
frequency selection, 541-542 
model of, 71-72 
statistical characterization of, 
535-542, 554 
type of fading exhibited, 72 
multipath component, 549 
multipath intensity profile, 539 
multipath phenomenon 
forms of, 532-533 
in a mobile radio environment, 
18 
nature of, 532 
multiple access 
basic types of, 513-514 
versus multiplexing, 513 
multiple-access interference (MAI), 
548 
multiple-access system 
goal of, 549 
interference, 548 
multiple-access techniques 
common feature of, 514 
defined as, 513-514 
ideas behind, 513, 514 
and shared communication 
resources, 513 
multiple-receiver combining 
techniques, 544 
multiplexed signal, 125-126 
multiplexed systems, 347 
multiplexer, 446 
multiplexing, 105 
definition of, 20 
of digital signals, 214 
types of, 20-21 
multiplier, 157, 646 


multi-pulse excited LPC, 551-552 
multitone jammer, 508 
multiuser communications, 512 
environment, 396 
types of, 559 
music 
as a source of information, 4 
structures of, 4 
musical signals 
and channel bandwidth, 4 
versus speech signals, 4 
mutual information, 584-585 
for continuous ensemble, 
593-597 
defined as, 596 
properties of, 585-587, 596 
in the Shannon model, 746 


N 
narrowband AM signal (wave), 
112-113 
narrowband FM signal (wave), 
111-112, 113 
narrowband FM waves, 112-113 
narrowband frequency modulation, 
111-113 
narrowband noise, 64 
characterization of, 64 
components of, 64, 65-66 
effects of, 64 
representation and coordinate 
system for, 67-68 
representation of, 64-66, 67-69 
narrowband noise analyzer, 64-66 
narrowband noise synthesizer, 
64-66 
narrowband phase modulator, 120 
narrowband process, 65-66 
National Television System 
Committee (NTSC), 6 
natural frequency, 160 
N-dimensional Euclidean space 
angles in, 312 
lengths of vectors, 312 
vectors in, 311, 312 
N-dimensional vector, 311 
near-end crosstalk (NEXT), 
279-280, 380 
nearest neighbor condition, 200 
near-far problem, 548, 549 
negative-going click, 150 
network, 10 
network, interconnected, 13 
network resources, 11, 14 


90 degrees rotational invariance, 
422-423 
noise 
absence of, 261 
calculations, 61 
in communications systems, 58 
definition of, 3 
effect of, 3 
minimizing the effects of, 157, 
260 
presence of, 589 
signals in, 322-326 
sources of, 3, 58 
as unwanted signals, 58 
noise analysis 
comparison of, 163-164 
at the receiver, 523 
noise calculations, 61 
noise channel, colored, 607-611 
noise enhancement, 283 
noise equivalent bandwidth, 
722-723 
` for bandpass filters, 723 
defined as, 723 
noise figure, 523-526 
noise-free estimates, 545 
noise jammer, barrage, 508 
noise jammer, partial-band, 508 
noise margin, 281 
noise masking, 234 
noise power, 3, 61 
noise power, average 
outputcalculation of, 151 
determining, 145-147 
noise-quieting effect, 147 
noise-to-mask ratio (N. 
noise vector ` 
covariance matrix of, 330 
particular realization of, 323 
statistical characteristics of, 330 
noisy receiver model, 130 
noisy resistor, 60 
noncoherent binary DPSK, 407 
noncoherent binary frequency-shift 
keying (FSK), 407, 413—414 
bit error rate for, 414 
consideration of, 413 
expressions for, 417 
noncoherent M-ary FSK detector, 
500 
noncoherent matched filter, 406 
noncoherent orthogonal 
modulation, 407—409 
noise performance of, 407 


), 234 


receiver for, 408 
special case of, 414 
noncoherent receiver, 405-406, 
409 
nondata-aided early-late delay 
(NDA-ELD) synchronizer, 458 
nondata~aided recursive algorithm, 
455 
nondata-aided synchronization, 
449 
nonflat channel, 542 
nonlinearity 
basic forms of, 126 
effects of, 126 
the presence of, 126 
nonlinear modulation process, 163 
nonlinear pre-emphasis and 
de-emphasis techniques, 157 
nonredundant coding, 421 
nonreturn-to-zero level encoder, 
359 
nonsystematic code, 656 
nonuniform quantizer, 202-203 
normalized Gaussian distribution, 
54-55, 56 
normalized transmission 
bandwidth, 164 
North, D. O., 27 
North American digital TDM 
hierarchy, 214-215 
Norton equivalent circuit, 60 
Noyce, Robert, 28 
NRZ binary data, 398 
NTSC system, 6 
null event, 704 
null-to-null bandwidth, 368, 721 
noumber-controlled oscillator 
(NCO), 458 
Nyquist, Harry, 27 
Nyquist bandwidth, 262 
Nyquist criterion, 262 
Nyquist interval, 186-187 
Nyquist rate, 186-187, 262 


o 

observable element, 321 
observation space, 324, 325-326 
observation vector, 321, 331 
octaphase-shift-keying, 365-366 
octet, 7 

offset QPSK, 362 

onboard switching, 516 
one-time pad, 747 

on-off level encoder, 384 


on-off signaling, 205 
open systems interconnection (OSI) 
reference model, 11 
optical communication, 29 
optical fiber 
advantages of, 17 
consists of, 17 
properties of, 17 
as a transmission medium, 17 
optical transmission system, 15 
optimization problem, 438-440 
optimum CAP receiver, 379 
optimum decision rule, 323-324 
optimum equalizer, 288 
optimum filter 
consists of, 378-379 
impulse response of, 250-251 
optimum information capacity, 610 
optimum in-phase filter, 378-379 
optimum linear receiver, 282-287 
interpretation of, 286 
and transmitter, 286 
optimum quadratic receiver, 
403-405 
optimum quadrature filter, 
378-379 
optimum quantization problem, 
199-201 
optimum receiver 
as a correlation receiver, 
326-327 
design of, 310 
detector part of, 328 
subsystems of, 326-327 
optimum receiver subsystems, 
326-327 
optimum threshold, 257 
orthogonal frequency-division 
multiplexing (OFDM), 
447-448 
applications of, 448 
techniques for broadcasting, 344 
use of, 447-448 
orthonormal basis functions, 311, 
315, 494 
as a desirable property, 440-441 
for MSK, 390 
shortcomings of, 441 
orthonormal matrix, 443 
oscillator, crystal-controlled, 120 
OSI model, 11-13 
outer conductor, 17 
output alphabet, 582 
output current, 4-5 
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output noise power, 523 
output signal-to-noise ratio 
of AM receiver, 136 
calculation of, 151 
versus carrier-to-noise ratio, 151 
definition of, 131, 145 
determining, 133 
equation for, 139, 497 
to evaluate, 135 
improvement factor in, 155-156 
increasing of, 155 
of a uniform quantizer, 197 
overmodulated, 90 


P 
packet switching 
network, 11 
principle of, 11 
pairwise error probability, 334 
PAM. See pulse amplitude 
modulation (PAM) 
PAM signal 
generation of, 188-189 
performance of, 191 
sampling of, 189 
transmission of, 191 
waveform of, 188-189 
PAM system, 191 
parallel encoding scheme, 676 
parallel-to-serial converter, 445 
parity bit, 7, 632 
parity-check equations, 634 
parity-check matrix, 634 
parity-check polynomial 
defined as, 644 
reciprocal of, 645 
partial-band noise jammer, 508 
partial-response signaling, 269, 
274-275 
partial-response signaling scheme, 
267 
achieved by, 274-275 
classes of, 275 
useful characteristics of, 275 
partitioning, 670- 
passband basis functions 
properties of, 434-436 
time variations of, 373 
passband data transmission, 344 
alternative techniques for, 465 
applications of, 344 
communication channel used for, 
344 
over nonlinear channels, 345 
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passband data transmission systems 
determining the bandwidth 
efficiency of, 348 
goal of, 346 
passband in-phase filter, 375, 378 
passband in-phase pulse, 375 
passband line code, 344 
passband modulation, 729 
passband pulse, 375 
passband quadrature pulse, 374 
passband signaling waveform, 
729 
passband transmission model, 
348-349 
path loss, 522 
pattern matching operation, 615 
PCM. See pulse-code modulation 
peak distortion, 288 
peak pulse signal-to-noise ratio 
defined as, 248-249, 250 
of a matched filter, 251 
peer process, 13 
percentage modulation, 90 
perception, 4 
perceptual coding, 9 
perfect security, 746-747 
periodicity, 718 
periodic signals, 717-718 
periodogram, 51 
persistence of vision, 5 
personal computers (PCs), 6, 6-7 
“phase and gain adjustors”, 550 - 
phase continuity, 388 
phase correction, 365 
phase decisions, 394 
phase demodulation, 492 
phase-difference computer, 364 
phase discriminator, 97 
phase distortion 
and the human ear, 99-100 
presence of, 99-100 
phase error 
defined as, 158 
effect of, 99 
phase-error generator, 459 
phase-locked loop, 121 
complexity of, 159-160 
components of, 157-160 
limitation of, 159-160 
loop filter in, 158 
model of, 158-160 
simplest form of, 159-160 
understanding, 157, 158 
use of, 157-160 


phase-locked loop demodulator, 
152 
and threshold extension capacity, 
154 
as a tracking filter, 154 
phase-locked loop theory, 157 
phase modulation (PM), 20, 108, 
729 
phase modulation schemes, 368 
phase nonlinearity, 127 
phase recovery, 450 
phase-recovery circuit, 345 
phase response, 719 
phase selectivity, 723-725 
phase sensitivity, 108 
phase-shift keying (PSK), 24 
and coherent systems, 490 
of phase modulation, 345 
represented by, 464 
signaling scheme, 344-345 
phase-shift keying (PSK) schemes, 
418 
phase spectrum, 715 
phase tree, 388 : 
phase-trellis, 388-389 
phasors, 532 
photocathode, 4 
photodetector circuit, 58-59 
physical layer, 13 
1/4-shifted DQPSK signals, 364 
11/4-shifted DQPSK symbols, 363 
1/4-shifted QPSK scheme, 363 
7/4-shifted QPSK signal 
demodulation of, 365 
residing in one of eight possible 
phase states, 362 
pictures 
and the human visual system, 4 
perception of, 4 
asa source of information, 4 
piecewise linear approximation, 
203 
Pierce, John R., 29 
pilot carrier, 99 
plain old telephone service (POTS), 
281-282 
plaintext, 742 
PM signal, 109 
PN sequence > 
correlation properties of, 506 
as an independent and identically 
distributed (iid) binary 
sequence, 496 
as a reference signal, 550 


pointer, 581 
point-to-point communication, 2~3 
Poisson’s sum formula, 718 
Poisson distribution, 59 
polar nonreturn-to-zero (NRZ) 
level encoder, 352 
polar nonreturn-to-zero (NRZ) 
signaling 
binary PCM system based on, 
253 
disadvantages of, 205-206 
polyvinylchloride (PVC) sheath, 16 
Positive-going click, 150 
postdetection filter, 143 
power, available, 61 
power control 
in CDMA systems, 549 
use of, 549 
power gain, 523 
of an antenna, 520 
concept of, 521 
definition of, 520 
power-limited channel, 3 
power spectra, 347 
power spectral density, 44-46, 
347 
and amplitude spectrum, 50-52 
frequency portions of, 155 
graphical summary of, 75~76 
properties of, 46-47 
of random process, 50, 52 
significance of, 45 
power spectrum, 4, 45 
power theorem, 520 
Poynting vector, 519 
PPM. See pulse-position 
modulation 
precoded duobinary scheme, 
270-271 
prediction, 9 
prediction filter, 228 
pre-envelope 
basic property of, 732 
defined as, 725, 730 
determining, 726 
quadrature components of, 374 
prefix code 
definition of, 575-576 
distinguished by, 577 
property of, 576 
prefix coding, 575 
prefix condition, 575 
premodulation low-pass filter, 
396-397 


preset threshold values, 277 
primary colors, 6 
represented by video signals, 6 
transmission of, 6 
primary rate, 214-215 
primitive BCH codes, 653 
primitive polynomial, 505 
principle of analysis by synthesis, 
551-552 
principle of rotational invariance 
illustration of, 330-331 
stated as, 330 
principle of superposition, 718 
principle of translational invariance 
application of, 331-332 
stated as, 331 
probabilistic code, 693 
probabilistic concepts, 703-707 
probabilistic decoder, 630 
probability 
axioms of, 704-706 
basic properties of, 705-706 
of bit error, 384 
of a correct decision, 357 
of error, 254, 409 
of symbol error, 258, 328-329, 
352 
probability, conditional, 706-707 
probability density function, 
67-68, 255, 594, 708-709, 
710 
probability distribution, 583 
probability of error, 328-329, 
497-499 
invariance of, 329-331 
for a noisy channel, 589 
union bound on the, 332-335 
probability of occurrence, 568 
probability of symbol error, 334 
determination of, 373 
evaluation of, 346-347 
formula for, 401 
for signal constellation, 337 
probability system, 704 
probability theory, 703, 705 
processing gain (PG) 
defined as, 229, 497 
produced by, 229 
product cipher, 749 
product modulator, 94, 98-99, 
111, 490 
propagation, 4 
propagation effects, 532-535 
propagation time delay, 516 


protocol 
of the Internet, 13-14 
types of, 13-15 
pseudo-noise (PN) sequence, 480, 
488 
consists of, 288 
generation and properties of, 
480 
as a training sequence, 288 
pseudo-random-ordered sequence, 
500 
PSK. See phase-shift keying 
psychoacoustic modeling, 9 
psychovisual redundancy, 9 
public-key cryptographic system, 
755 
public-key cryptography, 742, 
755-757 
public-key system, 759 
public switched telephone network 
(PSTN), 237, 420 
as an analog network, 420, 421 
distortion on, 286-287 
efficient use of, 425 
pulse, 5 
pulse-amplitude modulated signal, 
429 
pulse-amplitude modulation 
(PAM), 20, 188-191, 236 
definition of, 188 
and modulator design, 277 
and natural sampling, 188 
pulse-code modulation (PCM), 
193, 615 
advantages of, 217, 237 
bandwidth requirement of, 218 
basic condition of, 194 
cost of advantages, 217-218 
definition of, 201 
as a form of digital pulse 
modulation, 237 
performance of, 227 
as the preferred method, 20 
for speech coding, 229-230 
use of, 217-218, 560 
pulse-code modulation (PCM) link, 
210 
pulse-code modulation (PCM) 
receiver, 258 
pulse-code modulation (PCM) 
signal, 208 
pulse-code modulation (PCM) 
system, 218 
basic operations of, 201 
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characteristic of, 210 

description of, 201-209 

and interference, 210 

noise considerations in, 209-210 

operation of, 212 

performance influenced by noise, 
209 


“ pulse demodulator, 211 


pulse-duration modulation (PDM), 
20, 191-192, 236 
pulse-modulated signal, 237 
pulse modulation 
families of, 183 
forms of, 191-193, 237 
lossy nature of, 237-238 
method used to transmit, 211 
as source coding techniques, 
237 
standard digital form of, 20 
types of, 20 
pulse-modulation process 
incurred information loss of, 238 
loss of information and designer 
control, 238 
pulse modulation systems, 236 
pulse modulator, 211 
pulse noise jammer, 508 
pulse-position modulation (PPM), 
20, 192, 236-237 
pulse-position modulation (PPM) 
system 
versus frequency modulation 
system, 193 
noise analysis of, 193 
performance of, 193 
pulse shaping, 247 
pulse-shaping filter 
desirable properties of, 396-397 
Gaussian impulse response of, 
398 
pulse-shaping function, 373 
pulse-width modulation, 191 
punctured code, 676 
puncturing, 676 


Q 
QAM. See quadrature-amplitude 


modulation 
Q-channel, 97 
QPSK. See quadriphase-shift keying 
quadbits, 421 
quadratic receiver 
equation for, 405 
forms of, 405 
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quadrature-amplitude modulation 
(QAM), 97-98 
versus CAP, 369 
cross constellation, 371 
quadrature-amplitude modulation 
(QAM) constellations, 
369-370 
quadrature-amplitude modulators, 
433-434 
quadrature-carrier multiplexing, 
97-98, 354 
quadrature-carrier multiplexing 
system, 98 
quadrature channel, 408-409 
quadrature component, 93 
power spectral density of, 386 
properties of, 65-66 
role of, 93, 101 
quadrature modulation, 670 
quadrature noise component, 131 
quadrature null effect, 97 
quadrature-phase coherent 
detector, 97 
quadrature receiver 
channels of, 408 
using correlators, 405-406 
using matched filters, 405-406 
quadriphase-shift keying (QPSK), 
354, 354-361 
characterization of, 354-355 
error probability of, 356-358 
mode of operation, 425 
motivation for using, 508 
signal-space diagram, 354-355 
quadriphase-shift keying (QPSK) 
receiver, 360 
quadriphase-shift keying (QPSK) 
signal 
amplitude fluctuations, 362 
and binary PSK signal, 360 
commonly used constellations 
for, 362 
filtered, 361 
interference production, 396 
observations of, 360 
phase transitions of, 361 
power spectra of, 360-361 
quadriphase-shift keying (QPSK) 
transmitter, 359 
quality of service (QoS), 14 
quantization 
application of, 202-203 
and coding, 9 
purpose of, 8 
types of, 194-195 


use of, 195 
function of, 196 
types of, 220 
quantization noise, 195-197, 228 
designer’s control of, 209 
in delta modulation, 221 
as a function of time, 195 
and human ear perception, 9 
in PCM systems, 209 
quantization process, 193-195, 
236 
in the generation of a binary 
PCM wave, 193 
illustration of, 195 
nonlinear nature of, 198 
results of, 20 
quantization table, 8 
quantized excitation, 552 
quantized filter parameters, 552 
quantizer 
characteristics of, 194 
classes of, 615 
components of, 199 
as a signal compressor, 615 
types of, 194 
quantizer, nonuniform, 202-203 
quantizer input, 221 
quantum, 194 
quaternary system 
eye diagram for, 294-295 
output of, 276 


R 
radiation efficiency factor, 520 
radiation intensity, 519 
radiation-intensity pattern, 520 
radio communication link, 522 
radio communication system, 31 
radio link analysis, 517-523 
radio propagation 

in free space, 512 

in urban areas, 532-533 
radix, 570, 576-577 
raised cosine spectrum 

flat portion of, 264-265 

rolloff portion of, 264-265 
RAKE receiver, 549-550 

basic idea of, 549 

consists of, 549 

as a diversity receiver, 549 

techniques of, 559 
random binary sequence, 482 
random experiment 

description of, 703 

features of, 703 


random hopping, 500 
random interference, 31-32 
random process 
average power of, 610 
classes of, 75 
definition of, 33 
ensemble averages of, 41 
expectations of, 41 
in linear systems, 42—44 
mathematical definition of, 
32-33 
parameter of, 75 
properties of, 32 
through a linear time-invariant 
filter, 42-44 
random variable, 33, 594, 708-710 
definition of, 708 
description of, 708 
distribution of, 55-56 
expected value of, 711 
mean of, 711 
standard deviation of, 712 
variance of, 712 
random vectors, 594 
raster scanning, 4-5 
rate distortion function, 612-613, 
616 
application of, 616 
definition of, 613 
rate distortion theory, 611-613 
application of, 612 
main parameters of, 613 
and Shannon’s coding theorems, 
612 
Rayleigh’s energy theorem 
definition of, 251 
use of, 251-252 
Rayleigh distribution, 68-69, 70, 
74-75 
Rayleigh fading channel, 536, 541 
binary signaling over, 542-547 
performance of, 545 
received signal 
components of, 31-32 
mean value of energy, 543 
received signal point, 323 
received vector, 660 
receive filter, 259 
receiver 
of an analog communication 
system, 88-89 
assumptions of, 403 
de-emphasis in, 154-155 
model, 130 
moving-coil, 15-16 


noise performance of, 387 
as an optimum maximum 
likelihood detector, 436 
and preprocessing the received 
signal, 64 
receiver model, 130 
receiving antenna, 518 
reciprocity principle in antennas, 
521 
reconstruction filter, 187-188 
reconstruction levels, 194 
rectangular function, 262 
recursion theorem, 682 
recursive algorithm 
for phase recovery, 454 
for timing recovery, 457-458 
recursive symmetric convolutional 
(RSC) code, 675 
recursive Costas loop, 454 
convergence behavior of, 461, 
462 
operations of, 458 
phase-acquisition behavior of, 
459 
for phase synchronization, 454 
recursive early-late-delay 
synchronizer, 463—464 
redundancy 
addition of, 626 
basic forms of, 9 
controlled use of, 628 
redundant information, 227, 575 
Reed-Solomon codes, 654, 693 
Reeves, Alec, 27 
reference antenna 
definition of, 519 
as an isotropic source, 519 
reference signal, 557 
reflector antenna, 522 
regeneration, 208 
regenerative repeater, 208 
region of integration, 332 
regular-pulse excitation, 552 
regular turbo code, 692 
relative-frequency approach, 
703-704 
relative phase difference, 414 
relative phase shift, 532 
replication of delta function 
property, 717 
representation levels, 194 
reproduction quality, 5-6 
residual amplitude modulation, 112 
resolution, 6 
resolution of uncertainty, 568 


response time, 718 
reverse link 
versus forward link, 559 
subbands for, 547 
Rician distribution 
graphical presentation of, 70-71 
normalized form of, 71 
Rician fading channel, 536 
Rivest-Shamir-Adleman (RSA) 
algorithm, 759 
Rivest-Shamir-Adleman (RSA) - 
system, 757 
rms duration, 722 
robust system, 22 
rolloff factor, 265 
frequency response for, 265~266 
time response for, 265-266 
root mean square (rms) bandwidth, 
721 
rotated noise vector, 330 
router 
defined as, 13 
and host devices, 10 
primary purpose of, 10 
row vectors, 633 
RS-232 standard, 6-7 
RSA algorithm, 757-758 
RSA cryptoalgorithm, 758 
RSA trapdoor one-way function, 
758 


s 
sample functions, 32-33 
sample point, 32, 704 
sample space, 32, 704 
sampling, 201-202 
sampling period, 184 
sampling process, 236 
and digital signal processing and 
digital communications, 184 
in the generation of a binary 
PCM wave, 193 
and pulse modulation systems, 
184, 236 
use of, 184 
sampling rate, 184, 201 
sampling theorem, 201, 236 
for band-limited signals, 
186-187 
derivation of, 186-187 
essence of, 184 
of a pulse-modulation system, 
186-187 
recurrent nonuniform equivalent 
form of, 427 


INDEX 811 
satellite 
for communication, 29 
in geostationary orbit, 18 
services of, 18-19 
satellite channel 
capabilities of, 516 
coverage of, 18 
remote area access, 18 
satellite communications, 514-517 
frequency band for, 19 
global coverage, 512 
most popular frequency band 
for, 515 
as a type of multiuser 
communications, 512 
satellite communication system, 18 
design of, 517 
global coverage, 559 
rely on, 512 
unique system capabilities of, 
18-19 
scalar quantization 
form of, 194 
use of, 194 
scalar quantizer 
conditions for optimality of, 
198-201 
designing of, 198 
as a simple signal compressor, 
615 
scanning, 4-5, 4-6 
scanning spot, 5 
scattered beams, 71 
scatterers, 71-72 
scattering function, 539 
Schwarz’s inequality 
as a mathematical result, 
249-250 
proving, 313, 314 
SDMA, 516 
second flyback. See vertical 
retracesecond-order digital 
filter, 454-455 
second-order feedback system, 160 
secrecy, 745 
secret key 
versus public key, 759 
selection of, 745 
secret-key cryptoalgorithm, 751 
secret-key cryptography, 742-743 
secret-key system, 759 
secure channel, 743 
secure communications 
in a hostile environment, 479 
need for, 742 
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security of transmission, 490 
segments, 11 
See also packets 
separability theorem, 681 
sequential scanning 
of pictures, 4 
process of, 4-6 
serial-to-parallel converter, 445 
Shannon, Claudecapacity theorem, 
611 
and “The Mathematical Theory 
of Communication”, 27-28 
and the theoretical foundations 
of digital communications, 
27-28 
Shannon’s capacity theorem, 611 
Shannon’s fundamental bound for 
perfect security, 747 
Shannon’s information capacity 
theorem, 23-24, 433 
Shannon’s information theory, 617 
Shannon’s second theorem, 616 
Shannon’s third remarkable 
theorem, 616 
Shannon’s third theorem, 599 
Shannon limit, 602 
Shannon model of cryptography 
method of confusion, 749 ~ 
method of diffusion, 749 
methods of designing, 749 
shift parameters, 531 
shift register, 481 
Shockley, William, 28 
shot noise, 58-60 
sideband, upper and lower, 91 
side information, 230 
sigma-delta modulation, 222 
signal 
definition of, 3—4 
detection in noise, 322-326 
dimensions of, 3—4 
received versus transmitted, 2 
signal bandwidth, 3 
signal constellation, 322-323, 337 
as circularly symmetric, 335 
constructed from one- 
dimensional PCM symbols, 
429 
defining minimum distance of, 
335 
signal detection problem, 322 
likelihood function for, 405 
stated as, 323 
signal energy-to-noise spectral 
density ratio, 252 


signal fading, 532-533 
signal-flow graph, 665-666 
signaling binary information, 345 
signaling interval, 568 
signaling rate, 276 
signal parameters, 403 
signal power average, 3 
signal regeneration, 208 
signal-space analysis, 337 
signal-space dimensionality, 312, 
493 
signal-space representations 
of the interfering signal 
(jammer), 493 
of the transmitted signal, 493 
signals with unknown phase, 
403—406 
signal-to-mask ratio (SMR), 234 
signal-to-noise ratio 
basic definitions of, 3, 130-132 
at the device output, 524 
of an FMFB receiver, 153 
limitation of, 261 
of the source, 524 . 
signal-to-noise ratio gap, 432 
signal-to-quantization noise ratio, 
229 
signal transitions, 207 
signal transmission decoder, 326, 
349 
signal transmission encoder, 348, 
352 
signal variability, 530 
signal vector, 311 
simplex signals, 342 
signum function, 724 
sinc function, 262 
sine wave plus narrowband noise, 
69, 69-71 
single-key cryptography, 742 
single keyed oscillator, 384 
single-letter distortion measure, 612 
single sideband (SSB) modulation, 
98-100, 163 
basic operation in, 103 
definition of, 93 
in frequency-division 
multiplexing, 106 
single-sideband modulated signal, 
98-99 
single-tone FM signal, 112-113 
single-tone jammer, 508 
single-tone modulation 
and a narrowband FM signal, 
110 


and a wideband FM signal, 110 
sinusoidal carrier wave 
defined as, 90 
waveform of, 490, 492 
sinusoidal modulating signal 
(wave), 110 
sinusoidal modulation, 112-113 
sinusoidal wave, 88 
slicing levels, 277 
slope circuit, 121-122 
slope network, 143 
slope overload distortion, 220, 
221 
slow FH/MFSK signal, 501 
slow FH/MFSK system, 502 
slow-frequency hopping, 500-502 
smoothness, 223 
SNR ratio. See signal-to-noise 
ratiosoft-decision coding, 630 
soft-decision decoding, 669 
soft decisions, 630 
soft input-hard output, 693 
soft input-soft output, 693 
SONET, 15 
source code, 574 
type of, 575 
variability in lengths of, 579 
source code word, 21 
source coding, 574 
dissection of, 616-617 
for efficient communication, ~ 
567 
with a fidelity criterion, 611-612 
source-coding theorem, 574-575, 
612, 616 
average code-word length of, 
611 
in Shannon’s first theorem, 
574-575 
source decoder, 575 
source encoder, 21, 574 
functional requirements of, 574 
purpose of, 21 
spaced-frequency spaced-time 
correlation function, 538 
space diversity, 544-545 
space diversity technique, 546 
space-division multiple access 
(SDMA), 514 
space-time processor, 557 
spatial phenomenon, 534 
spatial sampling, 4-5 
spectral analysis, 110 
spectral content, 492, 493 
spectral decomposition, 443 


spectrally efficient modulation, 347 
spectrally efficient schemes, 347 
spectral nulls, 368 
spectral shaping, 348 
spectrum, 3, 4, 715 
spectrum despreading 
in demodulation, 491 
as a linear operation, 492 
spectrum spreading 
as a linear operation, 492 
and phase modulation, 491 
speech coding 
applications of, 229-230 
design philosophy of, 230 
at low bit rates, 229-230 
techniques for, 551 
speech communication process, 4 
speech-production process, 4 
speech signal, 4 
as bipolar, 6 
limits of, 16 
sphere packing, 599-600 
split-phase signaling, 207 
splitter, 282 
spontaneous fluctuations, 58-61 
spreading code 
with pseudo-random properties, 
490 
use of, 490 
spread spectrum 
communications, 508 
important attribute of, 488 
notion of, 488-490 
spread-spectrum communication 
system 
advantage of, 479 
rejection of interference, 479 
requirements of, 493 
spread-spectrum modulation 
definition of, 479-480 
for military applications, 480 
principles of, 480 
to provide multipath rejection, 
480 
secure communications of, 479, 
480 
signaling techniques known as, 
479 
spread-spectrum techniques 
as direct-sequence spread 
spectrum, 490 
in passband transmission, 490 
versus standard modulation 
techniques, 480 
square constellations, 369-370 


square law, 193 
SSB modulated signal, 99 
SSB modulation, 134 
standard modulation techniques, 
480 
state diagram, 657-660, 659 
state probabilities, 681 
static picture, 4 
stationary process, 33-34 
versus strictly stationary, 33 
various names for, 36 
statistical average, 711-714 
statistical expectation operator, 
711 
statistical regularity, 703 
step-size, 194 
step-size parameter, 225-226 
stereo multiplexing 
in FM radio broadcasting, 124 
as a form of frequency-division 
multiplexing, 124 
stochastic process, 32 
stop-and-wait automatic repeat 
request, 628 
stop-and-wait strategy, 628 
stream ciphers, 744-746 
operation of, 744 
used in, 745 
strictly stationary process, 35 
Strowger, A. B., 27 
Strowger switch, 27 
subframes, 552 
subnets, 13 
substitution cipher 
description of, 749-750 
use of, 749-750 
successive errors, 232 
sufficient statistics, 321 
sum-product algorithm, 691 
Sunde’s FSK, 381, 385-386, 388 
Sunde’s FSK signal, 386 
superhet, 128 
superheterodyne receiver, 27, 
128-129 
consists of, 128 
differences between AM and FM, 
129 
survivor paths, 662 
switching center, mobile, 530 
symbol, 2 
symbol energy-to-noise spectral 
density ratio, 502 
symbol error 
average probability of, 276 
conditional probability of, 333 
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symbol error probability 
versus bit error rate (BER), 
335-336 
to calculate, 357 
definition of, 209, 310 
evaluation of, 543 
formula for, 256-257 
to minimize, 310, 346 
as a ratio, 358 
simplification of, 335 
symbol rate, 501 
symbol shaping function 
defined as, 353 
energy spectral density of, 386, 
395 
symbol synchronization, 448 
symbol timing, 455, 458, 
463-464 
symbol timing recovery, 463-464, 
465 
symmetric modem configurations, 
421-425 
synchronization, 448-450, 493 
algorithmic (modern) approach, 
449 
basic modes of, 448 
classical approach to, 449 
implementation of, 449 
process of, 448 
as a statistical parameter 
estimation problem, 449 
in a TDM system, 212 
of transmitrer and receiver 
clocks, 212 
synchronization problem 
approaches for solving, 449 
solution to, 493 
synchronizing pulses, 5, 212 
synchronous demodulation 
quadrature null effect, 96 
synchronous optical network 
(SONET), 15 
syndrome 
calculation of, 646-648 
importance of, 635 
properties of, 635-636 
syndrome calculator, 647 
syndrome decoding, 635-636, 638, 
639 
syndrome polynomial, 647 
synthesis equation, 442-443 
synthesis filter, 552 
consists of, 552 
as part of the encoder, 551 
systematic block code, 632 
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system capacity 

definition of, 549 

description of, 718 

to maximize, 549 
system-dependent scaling factor, 

132-133 

system design objectives, 3 
system-memory time, 718 


T 
TDMA. See time-division multiple 
access 
telecommunications environment, 
3-7 
telegraph, 26 
telegraphic code, Baudot’s, 26 
telephone channel, 15-17, 287 
telephone circuit frequencies, 3 
telephone network, 27 
as a communication network, 
10 
first commercial service, 28 
primary purpose of, 15 
television network, 14 
television picture, 5-6 
television signals, 101-103 | 
modulation format of, 101-102 
as wideband signals, 7 
Telstar satellite, 29 
temporal autocorrelation function, 
284 
ternary code, 204 
t-error-correcting RS code, 654 
theorem of irrelevance, 321, 
321-322 
theory of error-control codes, 485 
theory of spectral analysis of 
random processes, 46 
thermal noise, 32, 60 
Thévenin equivalent circuit, 60 
3-dB bandwidth, 721 
three-level output, 267-268 
threshold effect, 137—138 
AM and FM, 164 
clicks heard in, 149-150 
definition of, 138, 149 
in an envelope detector, 138 
threshold extension, 154 
threshold reduction, 152 
time-averaged autocorrelation 
function, 42, 51 
time-bandwidth product, 721-722 
choice of, 398 
as a design parameter, 397 


time compression (TC) 
multiplexing 
scheme, 278-279 
use of, 277-278 
time diversity, 544-545 
time-division multiple access 
(TDMA), 211, 211-212, 513 
efficient system, 516 
as wireless communications 
systems, 547-550 
time-division multiplexing (TDM) 
concept of, 211 
defined as, 21, 105 
use of, 211-212 
time-domain description, 720 
time-flat channel, 542 
time-frequency mapping, 9 
time response, 266 
time-scaling property, 722 
time slot, 513 
time-to-frequency mapping 
network, 235 
time-varying phasor, 728 
time-varying transfer function, 536 
timing error, 264 
timing synchronization, 455 
Toeplitz property, 225 
toll connection, 16 
Tomlinson-Harashima precoding, 
430 
tone-modulation analysis, 119 
tracking, 493 
tracking filter, 154 
trade-offs, 602 
training mode, 290 
transceiver, 278 
transistor, 28 
transition matrix, 582 
transition probability, 582, 616 
transmission bandwidth 
defined as, 118 
instantaneous spreading, 499 
transmission delay, 260 
transmission path, 212 
transmission security, 490 
transmit filter, 259 
transmitted code vector, 660 
transmitted FH/MFSK signal, 500 
transmitted power 
definition of, 3 
primary communication 
resource, 92 
transmitted pulse amplitude, 253 
transmitted pulse shape, 261 


transmitted signal, 280-281 
transmitted signal energy per bit, 
258 
transmitted signal point, 322-323 
transmitted TV signal, 102 
transmitter 
of an analog communication 
system, 88-89 
by combining operations, 414 
location of, 2 
power limited, 597 
purpose of, 2 
use of pre-emphasis in, 154-155 
transmitting antenna 
function of, 518 
mounting of, 17-18 
as a point source, 519 
and power density, 521 
transorthogonal signals. See 
simplex signals 
transponder, 19, 514-515 
transponder channel, 515 
transposition cipher 
description of, 750 
use of, 750 
transversal equalizer, 286 
trapdoor one-way function, 755 
traveling-wave tube amplifier, 
516 
trellis, 657-660, 661-662 
trellis-coded modulation, 668-669 
trellis codes 
for band-limited channels, 
668-669 
design of, 669 
trellis coding 
as an error-control coding 
technique, 430 
as a forward-error correction 
scheme, 424 
trellis encoder, 425 
turbo codes , 
consist of, 674 
development of, 674 
performance of, 676-677 
properties of, 682, 693 
termination approaches of, 676 
turbo coding, 674 
turbo decoder 
basic structure of, 677-678 
complexity of, 684 
details of, 683 
turbo decoding, 677-680, 682-683 
turbo encoder, 683 


twisted-pair cable 
versus coaxial cables, 17 
consists of, 16 
susceptible to, 16-17 
uses of, 277 
2B1Q code, 281 
amplitude levels of, 317 
compared to other line codes, 
281 
desirable properties of, 281 
as the North American standard, 
281 
two-dimensional matched filter, 
378-379 
two-dimensional optimum receiver, 
378 
two-key cryptography, 742 
two-stage spectral analysis, 110 
two-step subspace procedure, 
556 
two-way transmission, 106 
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Ungerboeck, G., 28 
Ungerboeck codes, 670 
for 8-PSK, 670-672 
asymptotic coding gain of, 673 
unicity distance, 748 
uniform quantizer, 196 
union bound, 337 
illustration of, 333 
simplification of, 335 
as a useful upper bound, 
332-333 
use of, 401 
union of events, 333 
unipolar nonreturn-to-zero (NRZ) 
signaling, 205 
unipolar return-to-zero (RZ) 
signaling 
disadvantages of, 207 
feature of, 207 
uniquely decodable, 574 
unitary matrix, 443 
unit delay, 219 
unit-delay elements, 646 
universal curve, 118-119 
unmodulated carrier, 108 
up conversion, 104-105 
upconverter, 448 
uplink, 19, 514-515 
upper bound, 401 
upstream data transmission, 
281-282 
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coding, 423-424 
phase changes in, 422—423 
switching to QPSK mode, 424 
and trellis coding, 423-424, 425 
V.32 modem standardand 
alternative modulation 
schemes, 421 
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V.90 modem, 431 
van Duuren, H, C. A., 28 
Van Vleck, J. H., 27 
variable-length code, 574 
variable nodes, 685 
variance, 579 
VDSL. See very-high-rate digital 
subscriber lines 
vector product, 681 
vector quantizer 
advantage of, 615 
encoding process in, 615 
versus scalar quantizer, 615 
signal-to-quantization noise ratio 
for, 615 
vectors, 633 
vector space, 554-555 
Vernam cipher, 747 
very-high-rate digital subscriber 
lines (VDSL) 
advantages of, 446 
use of, 446 
very-large-scale integrated (VLSI) 
circuits 
development of, 28 
vestigial sideband (VSB) filter 
frequency response of, 102 
magnitude response of, 100-101 
vestigial sideband (VSB) modulated 
wave 
methods of generating, 100 
quadrature component of, 
102-103 
vestigial sideband (VSB) 
modulation, 100-101, 163 
vestigial sideband (VSB) shaping 
filter, 102 
vestigial sideband modulation 
definition of, 93 
and its role in commercial TV 
broadcasting, 101-102 
video bandwidth, 6 
video-on-demand, 9, 282 
video signal, 4~5, 6 
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virtual communication, 13 
Viterbi algorithm, 661-663, 670 
difficulty in the application of, 
662 
as a maximum likelihood 
decoder, 662 
as a maximum likelihood 
sequence estimator, 662 
VLSI. See very-large-scale 
integrated (VLSI) circuits 
vocal tract, 4 
voiceband modem, 420 
versus digital subscriber lines, 
446 
operational environment of, 447 
voice communication, 22 
voice effect, Donald Duck, 99-100 
voice signals, 99 
voice spectrum, 3 
voltage-controlled oscillator 
(VCO), 152-153, 157-158 
von Neumann, John, 28 
VSB. See vestigial sideband 
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water-filling interpretation, 
610-611 
water-filling solution, 438 
waveform, 21, 276 
of important line codes, 204-207 
in modulation, 490, 492 
waveform distortion, 102-103 
wavelength-division multiplexing 
(WDM), 21 
wave motion, 18 
weak signal suppression, 142 
weight vector, 554 
white Gaussian noise, 62 
identically distributed, 334 
process, 392 
white noise, 61—63 
autocorrelation function of, 
61-62 
characteristics of, 61-62 
mathematical properties of, 62 
power spectral density of, 61-63 
white noise process, 62 
wideband communication channels, 
218 
wideband FM signal, 118 
wideband frequency modulation, 
113-115 
wideband signal, 7 
wideband transmitted signal, 490 
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Wiener-Hopf equations, 224 
wired communications, 559-560 
wireless broadcast channels, 17-18 
wireless communications, 529-535 
adaptive antenna arrays for, 
553 
features of, $12 
goal of, 553 
major channel impairments of, 
553 
and mobility, 512 
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source coding for, 550-553 

as a type of multiuser 
communication, 512 

as a type of multiuser radio 
communication system, 
529-530 

versus wired communications, 
559-560 

wireless communication system 
mobility of, 559 


practical requirements of, 396 
problems using MSK, 396 
World Wide Web, 28-29 
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zero-mean white Gaussian noise 
process, 310 

zero state, 481 
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